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Abstract 

A  rigorous  system  for  rating  construction  contractor  performance  does  not  exist  for  the 
USAF  as  identified  by  the  Air  Force  Civil  Engineer  (USAF  CE),  Major  General  Del  Eulberg 
(Eulberg,  2007).  The  United  States  Army  Corp  of  Engineers  (USACE)  uses  DD  Form  2626  for 
contractor  performance  evaluation  and  contractor  selection.  The  objective  of  this  research  is  to 
strengthen  the  USAF  contractor  rating  system  by  exploring  USACE’s  use  of  DD  Fonn  2626. 

Using  data  from  DD  Form  2626,  statistical  analysis  was  conducted  to  determine  if  the 
measured  performance  sub-items  reflect  their  respective  performance  elements,  whether  the 
resulting  performance  elements  relate  to  the  overall  contractor  performance  rating,  and  finally,  if 
a  relationship  exists  between  the  overall  contractor  performance  rating  and  the  overall  project 
schedule  performance.  Two  hundred- fifteen  finalized  DD  Form  2626  were  evaluated  using 
various  statistical  analyses.  A  relationship  between  the  performance  elements  and  the 
contractor’s  overall  performance  rating  was  identified.  Two  of  nine  identified  performance 
elements  are  predictive  of  the  contractor’s  overall  rating.  The  DD  Form  2626  represents  a  good 
starting  point  to  meet  the  USAF  CE  intent.  However,  it  needs  standardized  instructions  and 
formatting  to  align  performance  items  and  elements  into  a  more  rigorous  system  for  rating 
contractor  performance. 
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AN  ANALYSIS  OF  CONSTRUCTION  CONTRACTOR  PERFORMANCE  EVALUATION 

SYSTEM 

I.  Introduction 

Ancient  armies  to  present  day  defense  services  all  have  at  least  one  thing  in  common — 

they  all  required  logistical  support.  In  the  early  Iron  Age  the  first  military  contractors  emerged 

as  merchants  who  organized  their  business  around  providing  military  supplies  to  the  local  armies 

on  a  regular  basis  (Gabriel  &  Metz,  1992).  The  same  basic  mechanism  applies  to  today’s 

military  procurement  programs.  However,  as  the  demand  for  contractor-provided  goods  and 

services  has  grown,  the  process  that  guides  the  solicitation,  evaluation,  and  selection  of 

contractors  has  matured.  The  Federal  Acquisition  Regulations  (FAR)  System  specifies  the 

procedures  and  policies  that  guide  the  procurement  of  services  and  materials  for  the  modern 

United  States  military.  The  FAR  System  exists  to  ensure  that  cost,  schedule,  and  quality 

requirements  are  fulfilled.  Key  to  this  procurement  effort  is  the  process  of  selecting  a  contractor. 

Contractor  selection  is  a  decisive  event  for  project  success.  It  corresponds  to  an 
interface  between  a  variety  of  construction  industry  clients  and  an  equally  varied 
array  of  construction  companies.  For  that  reason,  the  success  or  failure  of  the 
project  depends  on  this  interface,  because  it  is  the  magnifying  glass  used  to  look 
for  the  contractor  who  satisfies  the  project  objectives  in  the  best  way  (Alarcon  & 
Mourgues,  2002). 

The  current  criteria  used  by  the  United  States  Air  Force  (USAF)  to  select  a  contractor  is 
termed  best  value  source  selection  and  is  based  on  mission  capability,  cost,  past  performance, 
and  proposal  risk  (SAF/AQCP,  2008).  Put  simply,  the  Government  selects  the  contractor  that  is 
expected  to  provide  the  greatest  overall  benefit  in  response  to  the  defined  requirement 
(Department  of  Defense,  2005).  With  the  exception  of  cost,  all  the  criteria  evaluated  are  rated 
using  a  subjective  system  designed  to  indicate  the  USAF’s  overall  confidence  in  the  contractor’s 
future  performance.  Past  performance  is  considered  a  predictor  of  future  performance  (Wright, 
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1999).  The  current  system  is  not  without  its  challenges;  subjective  criteria  can  be  non-linear, 
uncertain,  and  imprecise  (El-Sawalhi,  Eaton,  &  and  Rustom,  2007). 

A  rigorous  system  for  rating  a  contractor’s  performance  for  use  in  best  value  source 
selection  does  not  yet  exist  for  the  USAF.  This  deficiency  was  specifically  identified  by  the  Air 
Force  Civil  Engineer,  Major  General  Del  Eulberg,  with  respect  to  military  construction  projects 
and  source  selection.  As  the  Air  Force  Civil  Engineer,  General  Eulberg  is  responsible  for  the 
installation  support  functions  at  166  Air  Force  bases  worldwide  with  an  annual  budget  of  more 
than  $17  billion.  General  Eulberg  identified  35  initiatives  in  an  effort  to  transform  the  civil 
engineering  career  field  into  a  more  efficient  and  effective  enterprise  (Eulberg,  2007).  One  such 
initiative  is  Transformation  Project  A-4.  The  purpose  of  this  initiative  is  to: 

•  strengthen  the  USAF  rating  system  of  contractor  performance 

•  standardize  performance  criteria  and  eliminate  inconsistencies  between  USAF 
MAJCOMs  and  bases 

•  ensure  best  value  is  achieved  for  the  government  by  establishing  a  system  of 
rewards  and  penalties  for  good  and  bad  contractor  performance  respectively 

Research  Objective 

The  objective  of  the  present  research  is  to  strengthen  the  USAF  rating  system  of 
contractor  performance  by  exploring  DD  Form  2626  Construction  Contractor  Performance 
Evaluation  used  by  the  United  States  Army  Corp  of  Engineers.  The  Army  Corp  of  Engineers 
uses  DD  Form  2626  to  evaluate  and  rate  construction  contractor  performance  and  then  uses  the 
evaluations  for  future  source  selections.  The  rest  of  the  chapter  provides  a  historical  review  of 
source  selection  and  the  evolution  from  lowest  bid  to  the  current  practice  of  best  value  source 
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selection.  This  chapter  focuses  on  the  motivation  behind  the  research  in  this  thesis,  the  problem 
statement  and  associated  research  questions,  as  well  as  the  methodology  employed  in  this  thesis 
research;  finally,  this  chapter  will  address  the  limitations  of  this  study  as  well  as  the  implications 
of  the  research. 

Background 

Historical  Problems  of  Lowest  Bid  Source  Selection 

Prior  to  the  1990s,  the  United  States  Government  selected  contractors  based  solely  on 
lowest  bid  price  (Gransberg  &  Ellicott,  1996).  Lowest  bid  contracting  assumed  that  project  plans 
and  specifications  that  were  complete  and  unambiguous,  allowed  for  the  price  to  be  the  sole 
competitive  factor  between  contractor  proposals.  Ultimately,  only  construction  costs  are 
considered,  excluding  the  costs  incurred  through  procurement,  project  management,  lost 
opportunities,  or  other  such  abstract  expenses.  It  became  clear  that  while  lowest  bid  contracting 
was  simple  and  straight  forward  to  implement,  it  often  resulted  in  higher  costs,  longer 
completion  time,  and  poorer  quality  (Feldman,  2006).  According  to  many  in  the  field,  awarding 
a  contract  solely  on  lowest  price  “poses  a  high  risk  to  the  client  because  there  is  an  increased 
possibility  of  financial  collapse  of  contractor,  bad  performance,  delay  in  completion,  time  and 
cost  overruns  and  so  on  (Wong,  Holt,  &  Harris,  2001).”  For  instance,  in  1992  the  US  Army  Corp 
of  Engineers  Europe  District  (EUD)  analyzed  four  problem  contracts  that  were  all  awarded  based 
on  lowest  bid  price  (Gransberg  &  Ellicott,  1996).  All  four  projects  were  behind  schedule  - 
project  completion  times  were  between  14  months  to  two  years  after  the  originally  scheduled 
completion  date;  all  were  above  the  US  Congress  authorized  program  amount  -  each 
experiencing  between  a  10%  up  to  a  30%  cost  growth;  quality  deteriorated  during  construction; 
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and  all  the  low  bids  were  submitted  by  marginal  firms  -  firms  with  long  histories  of  financial 
problems,  lack  of  experienced  and  skilled  management,  and  reputations  for  “buying-in”  to 
contracts.  Buying-in  is  defined  by  the  Federal  Acquisition  Regulation  (FAR)  as  “submitting  an 
offer  below  anticipated  costs,  expecting  to  increase  the  contract  amount  after  award  (Department 
of  Defense,  2005).”  Another  firm  lacked  experience  and  suffered  from  employment  of  unskilled 
managers  and  workers.  According  to  a  Gransberg  &  Ellicott  study,  an  investigation  of  each 
successful  offer  revealed  information  that  may  have  been  grounds  for  disqualification  from  the 
source  selection  had  a  different  procurement  strategy  been  employed  by  the  district.  “EUD's 
experience  indicated  that  minimum  levels  of  contractor  performance  rarely  met  customer 
expectations.  Increases  in  quality  were  generally  worth  a  corresponding  increase  in  cost 
(Gransberg  &  Ellicott,  1996).” 

In  a  separate  document,  Marcos  Feldman  identified  low-bid  contractor  selection  as  the 

source  for  construction  failure  in  the  Miami-Dade  County  (Feldman,  2006).  The  low-bid 

contracting  in  Miami-Dade  County  resulted  in  construction  delays  and  cost  overruns,  shoddy 

workmanship,  and  poor  construction  worker  health  and  safety. 

Low-bid  contracting  is  false  economy  as  the  initial  savings  from  price-based 
competition  are  erased  over  the  long-term  because  of  inferior  performance  leading 
to  additional  costs.  Low-bid  contracting  makes  flawed  assumptions,  encourages 
cost-cutting  and  underperformance,  and  does  nothing  to  screen  out  unscrupulous 
contractors.  (Feldman,  2006) 

Lowest  bid  acquisition  encourages  underperformance  by  the  contractors  who  are  in 
competition  with  other  low-bid  contractors.  Historically,  under  the  low-bid  acquisition 
methods,  contractors  would  underbid  a  contract  in  order  to  win  the  contract.  To  recover 
their  lost  profits,  contractors  would  then  use  substandard  materials,  poor  workmanship, 
and  take  great  risks  to  the  health  and  safety  of  their  laborers.  Lowest  bid  contracting  also 
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fails  to  account  for  contractor  performance  criteria  such  as  safety  records,  worker 
training,  schedule  compliance,  and  work  quality. 

Low-bidding  is  not  only  a  problem  for  the  United  States,  but  is  a  global  issue.  In  their 
article.  Predicting  Project  Performance  Through  Neural  Networks,  Cheung,  Wong,  Fung,  and 
Coffey  address  the  impact  of  the  1997  Asian  financial  crisis  and  the  affect  it  had  on  the  region’s 
construction  industry.  They  concluded  that  this  type  of  financial  environment  promotes  the 
submission  of  suicidal  bids  in  order  to  capture  work  opportunities,  especially  for  new  entrant 
construction  firms  (Cheung,  Wong,  Fung,  &  Coffey,  2006).  Such  suicidal  bids  can  result  in  poor 
quality  construction.  For  instance,  in  Hong  Kong,  prior  to  tenant  occupation,  two  newly 
constructed  multi-storied  housing  blocks  were  demolished  due  to  defective  foundation  work. 
While  it  is  not  conclusive  low-tender  value  caused  the  poor  construction  quality,  it  does  support 
the  argument  that  contractors  must  be  considered  both  on  their  technical  merit  as  well  as  their 
financial  solvency. 

The  analyses  of  contractor  performance  conducted  by  the  EUD,  Marcos  Feldman,  and 
Cheung  et  al.  demonstrated  that  lowest  bid  methods  for  acquisition  do  not  meet  traditional 
project  success  requirements  of  cost,  schedule,  and  quality  (Ling  &  Liu,  2004).  In  response  to 
demands  from  the  activist  procurement  policy  office  and  a  new  Pentagon  acquisition  reform 
operation,  Congress  passed  the  1994  Federal  Acquisition  Streamlining  Act  (FAS A)  and  the  1996 
Clinger-Cohen  Act.  The  intent  of  these  acts  was  to  streamline  acquisition  processes  and  reduce 
administrative  burdens  suffered  by  contracting  authority  offices  (Burman,  2000).  The  acts 
transitioned  the  government  from  lowest  bid  procurement  to  best  value  source  selection. 
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Best  Value  Source  Selection 


Following  the  FAS  A  and  Clinger-Cohen  legislation,  the  FAR  was  revised  Part  15  to 
promoted  best  value  procurements  over  lowest  bid  price  and  reshaped  the  way  government  does 
business  with  contractors  (Burman,  2000).  Government  agencies  created  partnerships  with 
contractors,  thus  streamlining  the  acquisition  process  that  is  focused  on  the  product  verses  the 
process.  One  of  the  first  statements  provided  in  the  FAR  is  the  Federal  Acquisition  System 
vision.  It  states: 

The  vision  for  the  Federal  Acquisition  System  is  to  deliver  on  a  timely  basis  the 
best  value  product  or  service  to  the  customer,  while  maintaining  the  public’s  trust 
and  fulfilling  public  policy  objectives.  Participants  in  the  acquisition  process 
should  work  together  as  a  team  and  should  be  empowered  to  make  decisions 
within  their  area  of  responsibility.  (Department  of  Defense,  2005) 

“Best  value  contracting  is  a  method  of  awarding  construction  contracts  in  which  bidders 

compete  on  the  basis  of  technical  and  managerial  merit,  past  safety  and  performance  records, 

qualification  of  craftsmen,  technical  innovation,  financial  health,  or  other  factors,  in  addition  to 

price  (Feldman,  2006).”  In  an  effort  to  provide  best  value  to  the  customer,  USAF  contractor 

source  selection  processes  consider  mission  capability,  cost,  past  performance,  and  proposal  risk 

when  evaluating  contract  proposals  (Wright,  1999).  Special  attention  is  paid  towards  a 

contractor’s  past  performance.  Section  1091  of  FASA  considers  the  past  performance  of  a 

contractor  to  be  “one  of  the  relevant  factors  that  a  contracting  official  of  an  executive  agency 

should  consider  in  awarding  a  contract.”  The  FAR  requires  contracting  officers  to  consider  past 

performance  for  all  competitively  negotiated  acquisitions  exceeding  $100,000  (Department  of 

Defense,  2005).  For  contracts  exceeding  $100  million  a  Performance  Confidence  Assessment 

Group  (PCAG)  conducts  a  comparative  past  performance  evaluation  in  order  to  “identify  the 

degree  of  performance  risk  associated  with  each  competing  offeror  (SAF/AQCP,  2008).” 
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The  methods  for  rating  contractors’  past  performance  however,  are  very  subjective 
(Wright,  1999).  Decision  makers  that  are  responsible  for  source  selection  and  who  are  bound  to 
subjective  rating  systems  as  those  found  in  the  AFFAR,  may  find  themselves  in  situations  where 
they  are  unable  to  effectively  defend  their  integrity  when  their  source  selection  decisions  are 
questioned.  In  fact,  integrity  of  the  source  selector  is  so  important  that  on  1  January  1997  the 
Procurement  Integrity  Act  went  into  effect  which  placed  restrictions  on  government  employees 
involved  with  the  selection  of  contractors  for  specified  government  programs.  For  example,  an 
employee  who  has  left  the  DoD  may  not  accept  compensation  from  the  affiliated  contractor  on  a 
$10  million+  DoD  contract  for  which  the  former  DoD  employee  performed  designated  services 
(48  C.F.R.  3.104-4(d)).  In  addition,  subjective  rating  systems  create  skepticism  regarding  the 
federal  government’s  best  value  procurement  process.  This  skepticism  has  caused  some 
companies  to  question  whether  fair  evaluations  will  be  made  during  source  selection.  “The  Best 
Value  process  has  also  caused  them  to  revisit  bid  decisions  in  the  context  of  return-on- 
investment  and  risk  (Mickaliger,  2001).”  When  selection  process  is  not  rigorous  or  is  highly 
subjective,  and  when  decision  makers  deliberately  compromise  their  integrity  leading  to  fairness 
concerns  from  participating  contractors,  the  overall  process  of  best  value  source  selection  is 
negated.  Rather  than  selecting  the  best  contractor  for  the  best  value,  the  government  is  likely  to 
hire  an  unqualified  contractor — a  decision  that  can  result  in  an  over-budget,  over-schedule 
executed  project. 

Motivation  for  this  Research  Effort 

Best  value  is  continually  encouraged  as  a  means  for  federal  acquisition  of  products  and 
services;  defining  “best  value”  is  subjective,  vague,  and  non- standardized  throughout  the 
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government.  A  contractor’s  past  perfonnance  is  a  primary  consideration  in  best  value  source 
selection.  Past  performance  is  considered  a  predictor  of  future  performance.  The  method  used 
to  identify  the  USAF’s  overall  confidence  in  a  contractor’s  future  perfonnance  is  a  subjective 
rating  system  (Wright,  1999).  However,  subjective  criteria  are  non-linear,  uncertain,  and 
imprecise,  making  the  task  of  contractor  selection  challenging  (El-Sawalhi,  Eaton,  &  and 
Rustom,  2007).  A  rigorous  system  for  rating  or  determining  a  contractor’s  future  performance 
does  not  exist  for  the  USAF. 

The  United  States  Army  Corp  of  Engineers  (USACE)  is  an  organization  comprised  of 
military  and  civilian  engineers,  scientists  and  other  professions  whose  mission  is  to  provide 
quality,  responsive  engineering  services  to  the  nation.  They  are  responsible  for  design  and 
construction  management  support  for  Defense  and  federal  agencies,  to  include  the  USAF.  As 
part  of  their  construction  management  process,  the  USACE  completes  a  contractor  performance 
evaluation  on  all  construction  projects.  The  performance  evaluations  are  captured  and 
standardized  on  DD  Form  2626  (see  Appendix  A).  Once  this  evaluation  is  finalized  and 
approved,  it  is  stored  in  a  centrally  managed,  Department  of  Defense  database  called  CPARS. 
CPARS  stands  for  Contractor  Performance  Assessment  Rating  System.  The  contractor 
performance  evaluations  are  then  available  for  reference  when  USACE  project  managers  are 
making  determinations  about  a  contractor’s  qualification  for  future  construction  projects.  The 
obvious  question  that  arises  is  whether  the  construction  performance  evaluation  process  used  by 
the  USACE  constitutes  a  reliable  and  defensible  method  for  rating  a  contractor.  Can  the 
contractor  performance  evaluation  data  collected  on  DD  Form  2626  predict  a  contractor’s  future 
performance?  If  the  construction  performance  evaluation  process  used  by  the  USACE  is  reliable 
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and  valid,  can  it  be  adopted  by  the  USAF  civil  engineers  as  a  method  for  rating  and  predicting 
contractor  performance? 

Problem  Statement 

The  objective  of  this  research  is  to  analyze  DD  Form  2626,  the  USACE’s  method  for 
evaluating  construction  contractors,  in  order  to  determine  if  the  form  is  reliable,  valid,  and 
appropriate  for  implementation  by  the  USAF  civil  engineers  for  evaluating  and  rating  contractor 
performance.  The  research  will  analyze  data  from  DD  Form  2626  to  determine  if  any  of  the 
perfonnance  items  are  correlated  to  a  contractor’s  overall  perfonnance  rating.  The  research  will 
detennine  if  a  relationship  exists  between  the  project  schedule  perfonnance  and  the  contractor’s 
overall  performance. 

Research  Questions 

A  thorough  analysis  of  construction  contractor  performance  evaluations  will  focus  on 
answering  the  following  specific  research  questions: 

1.  Do  the  DD  Form  2626  performance  elements  reflect  their  respective  performance  items? 

2.  Do  the  performance  items  as  they  are  appropriately  aggregated  into  performance 
elements  predict  the  contractor’s  overall  perfonnance  rating? 

3.  Is  the  contractor’s  overall  performance  rating  and  the  overall  project  schedule 
performance  related? 

Figure  1  is  a  conceptual  model  of  the  performance  relationships  that  will  be  evaluated  in  this 
research. 
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Performance  Items 

Performance  Elements 

QC.a 

Quality  of  service 

QC.b 

Adequacy  of  CQC  plan 

QC.c 

Implementation  of  CQC  plan 

QC.d 

Quality  of  QC  documentation 

QC.e 

Storage  of  materials 

QC.f 

Adequacy  of  materials 

RQ#1 

QC.g 

Adequacy  of  submittals 

— ^  QC  -  Quality  Control 

QC.h 

Adequacy  of  QC  testing 

QC.i 

Adequacy  of  as-builts 

QC.j 

Use  of  specified  material 

QC.k 

Identification/correction  of 

deficient  work  in  a  timely 

manner 

EM.a 

Cooperation  and  responsiveness 

EM.b 

Management  of  resources/ 

personnel, 

EM.c 

Coordination  and  control  of 

subcontractors) 

EM.d 

Adequacy  of  site  clean  up 

EM.e 

Effectiveness  of  job-site 

RQ#1 

supervision 

— ^  EM  —  Effectiveness  of  Management 

EM.f 

Compliance  with  laws  and 

regulations 

EM.g 

Professional  conduct 

EM.h 

Review/resolution  of 

subcontractor  issues 

EM.i 

Implementation  of  subcontractor 

plan 

TP.  a 

Adequacy  of  initial  progress  schedule 

TP.b 

Adherence  to  approved  schedule 

TP.c 

Resolution  of  delays 

TP.d 

Submission  of  required  documentation 

RQ#1 

TP.e 

Completion  of  punchlist  items 

— ^  TP  -  Timely  Performance 

TP.f 

Submission  of  updated  and  revised 

progress  schedules 

TP.g 

Warranty  response 

CLS.a 

Correction  of  noted  deficiencies 

CLS.b 

Payrolls  properly  completed  and 

submitted 

RQ#1 

CLS.c 

Compliance  with  labor  laws  and 

— p  CLS  —  Compliance  with  Labor  Standards 

regulations  with  specific  attention  to 

the  Da vis-Bacon  Act  and  EEO 

requirements 

CSS.a 

Adequacy  of  safety  plan 

RQ#1 

CSS.b 

Implementation  of  safety  plan 

— ^  CSS  —  Compliance  with  Safety  Standards 

CSS.c 

Correction  of  noted  deficiencies 

Figure  1.  Relationship  Conceptual  Model 
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Overall 

Performance 


RQ  #3 


Overall 

Project  Schedule 
Performance 


Methodology 

Project  performance  data  was  collected  from  finalized  DD  Forms  2626  Performance 
Evaluation  (Construction)  for  Fiscal  Year  (FY)  1998  to  FY  2008.  Only  finalized,  completed, 
and  approved  DD  Forms  2626  were  used  in  this  research.  All  DD  Forms  2626  used  in  this 
research  are  from  a  regional  US  ACE  office  with  construction  projects  ongoing  in  five  states. 

A  principle  component  analysis  was  conducted  on  the  thirty-three  performance  items 
from  the  DD  Form  2626.  The  principle  component  analysis  was  used  to  determine  if  the 
observed  variables  are  condensable  into  a  smaller  set  of  variates  (factors)  with  minimal  loss  of 
information  (Hair,  Black,  Babin,  Anderson,  &  Tatham,  2006).  After  condensing  each  of  the 
thirty-three  performance  items  into  performance  factors,  logistical  regression  was  used  to 
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identify  which  performance  factors  correlated  with  the  overall  performance  rating.  The  data  set 
was  divided  into  two  random  groups.  The  first  group  of  data  was  used  to  determine  the 
relationship  the  performance  factors  have  with  the  overall  performance  rating.  The  second  group 
of  data  was  used  to  test  the  ensuing  model  for  validity.  Finally,  using  the  first  group  of  data,  the 
overall  performance  rating  was  correlated  with  the  overall  project  schedule  performance  to 
determine  if  a  relation  exists. 

[.imitations 

The  use  of  DD  Form  2626  data  placed  several  unavoidable  limitations  on  this  research 
that  must  be  identified.  First,  the  sample  size  used  to  analyze  the  data  on  DD  Form  2626,  while 
by  definition  is  a  large  sample  size  it  is  not  large  enough  to  perform  statistical  analysis  on  each 
performance  item.  The  second  limitation  is  that  it  is  a  cross  sectional  study;  the  evaluations  are 
all  completed  at  the  end  of  the  construction  project  -  one  point  in  time.  Rater  anonymity  of  each 
DD  Form  2626  is  the  third  limitation  of  this  study.  According  to  a  resident  engineer  for  the 
USACE,  the  name  identified  on  each  DD  Form  2626  as  the  evaluator  is  the  name  of  the  resident 
engineer.  The  resident  engineer  has  oversight  on  all  projects  within  his  or  her  office.  The 
project  manager  will  complete  the  DD  Form  2626  and  the  resident  engineer  will  review  the 
evaluation  for  correction  before  signing  his  or  her  name  to  the  final  evaluation.  Therefore,  since 
it  isn’t  possible  to  identify  each  rater  for  each  data  point,  calculation  of  individual  rating 
differences  between  raters  is  not  possible.  The  fourth  limitation  to  this  study  is  the  time  period 
for  sampling.  The  USACE  has  been  involved  with  construction  project  management  since  its 
inception  16  March  1802;  the  data  analyzed  in  this  research  covers  a  period  between  the  years 
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2000  to  2008.  The  fifth  limitation  of  the  data  is  its  inability  to  be  generalized;  all  215  DD  Form 
2626  attained  from  the  USACE  were  from  a  regional  office  that  supports  the  mid-west  region. 

Implications 

The  results  of  this  study  will  identify  if  relationships  exist  between  the  performance 
items,  the  contractor’s  overall  performance  rating,  and  the  overall  schedule  performance  of  a 
contractor  as  evaluated  using  the  DD  Form  2626.  If  a  relationship  exists  between  the 
performance  items  and  the  overall  contractor  rating,  the  data  analysis  will  provide  project 
managers  and  contracting  officers  a  mathematical  model  for  predicting  contractor  performance 
that  can  be  employed  during  the  interim  evaluation  of  construction  contractors.  The  predictor 
model  will  provide  the  government’s  project  managers  the  ability  to  identify  deficient 
contractors  early  in  the  construction  phase  and  possibly  prevent  schedule  delays  and  budget 
overruns  that  would  result  in  below  satisfactory  contractor  performance.  Additionally  the 
performance  predictor  model  could  enhance  the  existing  rating  system  of  construction 
contractors.  The  results  should  identify  objective  project  indicators  that  quantitatively  identify  a 
contractor’s  future  performance  as  either  above  average  (good)  or  below  average  (poor). 

If  relationships  do  not  exists  between  the  performance  items,  the  contractor’s  overall 
performance  rating,  and  the  overall  cost  and  schedule  performance,  then  this  research  will  help 
identify  deficiencies  in  the  current  contractor  evaluation  process  and  the  application  of  DD  Form 
2626. 
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Thesis  Organization 

Chapter  1  establishes  the  framework  for  the  study  by  describing  the  impacts  of  lowest-bid 
source  selection,  the  subsequent  implementation  of  best  value  source  selection,  and  the 
consequences  of  selecting  contractors  based  on  subjective  criteria.  A  brief  history  of  government 
source  selection  is  described  starting  with  lowest-bid  source  selection  and  followed  by  best  value 
source  selection.  The  problem  statement  and  research  questions  identify  the  focus  of  this  study: 
to  determine  if  relationships  exist  between  the  performance  items,  the  contractor’s  overall 
performance  rating,  and  the  overall  schedule  performance.  If  a  relationship  exists  between  the 
overall  contractor’s  rating  and  the  performance  items,  a  statistical  model  can  then  be  constructed 
for  predicting  contractor  performance  using  available  construction  contractor  performance 
evaluation  data.  Chapter  2  examines  the  literature  on  best  value  source  selection,  current  USAF 
source  selection  practices,  USACE’s  construction  contractor  evaluation  process,  existing 
methods  for  predicting  contractor  performance,  and  project  performance  measures  and  criteria. 
Chapter  3  describes  the  statistical  methodology  used  to  identify  correlations  between  the  overall 
contractor’s  performance  and  the  individual  evaluations  of  a  contractor’s  performance  items  and 
to  develop  a  linear  regression  model  for  predicting  contractor  performance.  It  identifies  the 
performance  items  and  how  the  each  item  is  consolidated  into  performance  elements.  Chapter  4 
outlines  the  results  and  analysis  of  the  study.  Chapter  5  provides  the  discussion  and  conclusions 
gained  from  the  study  along  with  recommendations  for  future  research. 
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II.  Literature  Review 


Chapter  Overview 

This  chapter  investigates  the  current  literature  and  research  regarding  contractor 
performance  predictor  models.  The  purpose  is  to  understand  the  framework  that  governs 
contractor  selection  and  has  led  to  the  development  of  various  construction  contractor  predictor 
models.  The  following  topics  will  be  investigated  here:  best  value  source  selection;  the  current 
contractor  selection  methods  employed  by  the  United  States  Air  Force  (USAF);  United  States 
Army  Corp  of  Engineer’s  (USACE’s)  construction  contractor  evaluation  process;  contractor 
performance  predictor  models;  and  study  performance  metrics.  This  will  build  the  foundation 
for  the  methodology  used  to  test  the  reliability  and  validity  of  DD  Form  2626  as  an  appropriate 
tool  for  rating  and  predicting  construction  contractor  performance. 

Best  Value  Source  Selection 

Best  value  source  selection  is  the  framework  that  governs  contractor  selection  for  the 
U.S.  Government.  Source  selection  is  “the  process  wherein  the  requirements,  facts, 
recommendations,  and  government  policy  relevant  to  an  award  decision  in  a  competitive 
procurement  of  a  system/project  are  examined  and  the  decision  made  (Defense  Acquisition 
University,  2005).”  Otherwise  stated,  source  selection  is  the  process  of  “choosing  an  offeror  for 
the  contract  award  on  the  basis  of  integrated  assessment  of  non-cost  factors  as  well  as  cost  or 
price  (Wright,  1999).”  Best  value  source  selection  is  the  process  of  acquisition  that,  in  the 
Government’s  estimation,  provides  the  “greatest  overall  benefit  in  response  to  the  requirement 
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(Department  of  Defense,  2005).”  The  FAR  encourages  contracting  officers  to  use  best  value 


source  selection. 

When  the  government  employs  best  value  source  selection,  it  provides  itself  with  the 
opportunity  to  select  a  higher  priced  proposal  as  a  tradeoff  for  other  important  non-cost  factors 
such  as  quality,  safety  record,  or  past  performance  to  name  a  few.  A  tradeoff  is  defined  by  the 
Acquisition  Glossary  as  “selection  among  alternatives  with  the  intent  of  obtaining  the  optimal, 
achievable  system  configuration.  Often  a  decision  is  made  to  opt  for  less  of  one  parameter  in 
order  to  achieve  a  more  favorable  overall  system  result  (Defense  Acquisition  University,  2005).” 
According  to  FAR  15. 101 -1(c),  paying  additional  cost  for  an  acquisition  is  merited  with  the 
decision  maker(s)  perceive  higher  benefits  to  the  Government  with  the  higher  priced  proposal. 

One  such  benefit  is  the  satisfactory  or  above  average  past  performance  of  a  contractor. 
For  source  selection  purposes,  past  performance  information  is  considered  relevant  and 
important  information.  Under  the  FAR,  subpart  42.15  Contractor  Performance  Information, 
typical  criteria  used  to  evaluate  a  contractor’s  past  performance  includes 

•  the  contractor’s  record  of  conforming  to  contract  requirements  and  to  the 
standards  of  good  workmanship 

•  the  contractor’s  record  of  forecasting  and  controlling  costs 

•  the  contractor’s  adherence  to  contract  schedules 

•  the  contractor’s  history  of  reasonable  and  cooperative  behavior  and  commitment 
to  customer  satisfaction  (Department  of  Defense,  2005) 

The  evaluation  of  a  contractor’s  past  performance  includes  the  ability  of  the  contractor  to  meet 
cost,  schedule,  and  quality  requirements  that  are  satisfactory  to  the  customer. 
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The  source  selection  process  can  be  broken  down  into  six  steps.  In  the  first  step  of  source 
selection,  the  government  identifies  technical  differences  between  proposals  measured  against 
the  criteria  established  in  the  Request  for  Proposal  (RFP)  or  Statement  of  Work  (SOW).  In  the 
second  step  of  the  selection  process,  the  selection  officials  determine  the  potential  impact  to 
agency  operations  of  each  technical  difference.  “The  government  assigns  a  positive  or  negative 
impact  statement  on  economic  benefits  clearly  attributable  to  increased  productivity,  service 
delivery  to  the  public,  mission  effectiveness,  and/or  other  unique  approaches  (Mickaliger, 

2001).”  In  the  third  step,  similar  technical  differences  are  consolidated  and  those  with  limited 
impact  on  source  selection  are  eliminated.  In  the  fourth  step,  source  selection  officials  apply 
predetermined  weight  factors  for  each  remaining  discriminator  relative  to  their  favorable, 
neutral,  or  unfavorable  influence  on  the  impact  areas.  Documentation  of  analytical  methodology 
applied  and  its  associated  rationale  occurs  in  step  five.  This  includes  documentation  of  data  used 
for  required  calculations  as  well  as  identifying  and  defining  all  assumptions  used  in  the  analysis. 
The  final  step  in  the  source  selection  process  is  to  document  the  tradeoff  process  used  to 
determine  the  “quantified  proposal  discriminators  and  the  relative  value  of  the  proposal  by 
considering  the  non-quantified  discriminators  (Mickaliger,  2001).”  Using  their  analytical 
process  for  determination,  selecting  officials  make  their  final  decision  and  award  the  contract  to 
the  ‘most  successful’  offeror.  The  development  of  contractor  performance  predictor  models  are 
a  result  of  steps  four  through  six  of  the  contractor  selection  process.  By  employing  rigorous 
mathematical  models,  decision  officials  attempt  to  identify  the  most  qualified,  technically- 
acceptable,  construction  contractor  using  the  least  subjective,  but  most  standardized  and  fair 
system  of  elimination.  The  intent  of  this  research  is  to  identify  a  method  the  United  States  Air 
Force  (USAF)  decision  makers  can  use  when  rating  and  predicting  construction  contractor 
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performance.  The  six-step  contractor  selection  process  identified  by  Mickaliger  serves  as  the 
foundation  that  governs  current  USAF  source  selection  practices.  The  next  section  discusses  in 
detail  the  specific  methods  employed  by  the  USAF  for  source  selection. 

Current  USAF  Source  Selection  Practices 

To  support  the  Civil  Engineer  Transformation  Initiative  A-4,  the  intent  of  this  research  is 
to  strengthen  the  USAF  rating  system  of  contractor  performance,  standardize  performance 
criteria  and  eliminate  inconsistencies  between  USAF  MAJCOMs  and  bases,  and  to  ensure  best 
value  is  achieved  for  the  government  by  establishing  a  system  of  rewards  and  penalties  for  good 
and  bad  contractor  performance  respectively.  Research  into  the  current  USAF  source  selection 
practices  must  be  conducted  to  identify  a  more  rigorous  USAF  contractor  performance  rating 
system  that  can  be  used  to  predict  future  contractor  performance. 

In  1998,  Expert  Choice,  Inc.  and  Battelle  Memorial  Institute  wrote  a  report  addressing 
protest  proof  source  selection.  Expert  Choice,  Inc.  is  a  software  and  technology  services 
company  that  focuses  on  providing  collaborative  decision  support  solutions.  “The  company’s 
products  allow  users  to  structure  and  measure  objectives  and  alternatives,  perform  sensitivity 
analysis,  identify  funding  levels,  align  resources,  source  and  select  vendors,  identify  and 
prioritize  risks,  and  perform  gap  analysis  (BusinessWeek,  2008).”  Battelle  Memorial  Institute  is 
a  non-profit  organization  dedicated  to  the  furthering  international  science  and  technology.  Their 
key  areas  of  concern  are  energy,  health  and  life  sciences,  national  security,  laboratory 
management,  and  education.  In  their  1998  report,  they  state  that  “choosing  the  winning 
contractor,  selecting  the  best  product,  or  picking  a  new  supplier  involves  an  assessment  of  how 
well  each  alternative  contractor,  product,  or  supplier  satisfies  the  objectives  or  criteria  being 
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considered  (Expert  Choice,  Inc.,  1998).”  When  selecting  contractors  for  acquisitions  valued  at 


one  million  dollars  or  greater,  the  USAF  employs  a  color  rating  system  which  indicates  how  well 
the  evaluation  standards  are  met  by  each  offeror.  Acquisitions  using  the  simplified  acquisition 
procedure  are  exempt  from  applying  this  procedure  (SAF/AQCP,  2008).  The  color  ratings  are 
summarized  in  Table  1. 


Table  1.  USAF  Contractor  Performance  Color  Rating  System  (Expert  Choice  Inc.,  1998) 

COLOR 

RATING 

DEFINITION 

Blue 

Exceptional 

Exceeds  specified  performance  or  capability  in  a  beneficial  way  to  the  Air 
Force  and  has  no  significant  weakness 

Green 

Acceptable 

Meets  evaluation  standards  and  any  weaknesses  are  readily  corrected 

Yellow 

Marginal 

Fails  to  meet  evaluation  standards;  however,  any  significant  deficiencies 
are  correctable 

Red 

Unacceptable 

Fails  to  meet  a  minimum  requirement  of  the  RFP  and  the  deficiency  is 
uncorrectable  without  a  major  revision  of  the  proposal 

Evaluation  teams  are  discouraged  from  using  numerical  weights  because  it  implies  it  is  possible 
to  detect  small  differences  in  technical  merit  that  would  allow  an  evaluation  team  to  differentiate 
between  solicitations  (AFFARS,  AA-304(b)(c)).  However,  according  to  an  article  provided  by 
Expert  Choice,  Inc.,  “. .  .it  is,  in  fact,  possible  for  a  team  to  differentiate  between  small 
differences  on  any  factor  being  judged  if  the  right  measurement  method  is  used.  It  does  not 
matter  whether  the  factor  is  tangible  or  intangible,  whether  the  data  is  soft  or  hard  (Expert 
Choice,  Inc.,  1998).”  The  document  highlights  how  difficult  it  is  to  summarize  a  bidder’s 
performance  and  determine  ‘best  value’  when  the  results  look  like  Table  2. 
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Table  2.  Sample  USAF  Bidder  Rating  (Expert  Choice  Inc.,  1998) 

CRITERION 

BIDDER  A 

BIDDER  B 

1 

Blue  (Exceptional) 

Green  (Acceptable) 

2 

Green  (Acceptable) 

Yellow  (Marginal) 

3 

Yellow  (Marginal) 

Yellow  (Marginal) 

4 

Yellow  (Marginal) 

Green  (Acceptable) 

Expert  Choice,  Inc.  argued  that  “numbers  are  required  to  explain  how  much  better  one  value  is 
than  another  (Expert  Choice,  Inc.,  1998).”  In  the  event  of  an  audit,  both  narrative  and  numeric 
arguments  for  source  selection  are  required  to  justify  any  and  all  contractor  selections. 

This  color  approach  is  widely  used  in  source  selection,  performance  evaluation,  site 
selection,  and  funds  allocation.  Though  it  appears  systematic,  logical,  and  rational,  the  reality  is 
that  the  color  system  can  be  misleading,  inappropriate,  or  even  wrong  resulting  in  sound  bases 
for  source  selection  protests.  A  weighting  system  is  typically  employed  to  combat  the  inequality 
of  numeric  ratings  scales  when  evaluating  various  performance  factors.  However,  “assigning 
weights  is  as  unreliable  as  assigning  scores  on  a  1-10  rating  scale.  It  is  difficult,  if  not 
impossible,  to  force  all  evaluators  to  use  the  same  scale  the  same  way  (Expert  Choice,  Inc., 
1998).”  Consider  Gary  D.  Holt,  et  al.  ’s  survey  of  factors  that  influence  a  construction  client’s 
choice  of  contractors.  Of  the  30  factors  surveyed,  none  received  100%  congruent  ratings  and 
weighting  by  the  53  clients  surveyed  (Holt,  Olomolaiye,  &  Harris,  1994).  Their  survey 
demonstrated  that  every  person  involved  in  contractor  selection  not  only  prioritizes  various 
contractor  performance  factors  differently,  but  they  also  place  varied  values  on  each  of  those 
performance  factors.  Hence,  rather  than  trying  to  develop  a  standardized  contractor  performance 
rating  system,  many  researchers  instead  looked  into  models  that  would  predict  contractor 
performance.  Identifying  contractor  performance  predictor  models  aligns  with  the  intent  of  this 
research  which  is  to  identify  a  method  for  use  by  USAF  decision  makers  when  rating  and 
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predicting  construction  contractor  performance.  Rather  than  developing  a  rating  system  or 
predictor  model  from  scratch,  this  research  will  examine  the  USACE’s  construction  contractor 
evaluation  process  for  reliability  and  validity.  If  reliability  and  validity  exists  with  their  process, 
then  the  USACE’s  construction  contractor  evaluation  process  will  be  recommended  to  the  USAF 
civil  engineer  community  as  a  means  for  strengthening  the  USAF  rating  system  of  contractor 
performance,  standardizing  performance  criteria  and  eliminating  inconsistencies  between  USAF 
MAJCOMs  and  bases,  and  ensuring  best  value  is  achieved  for  the  government  by  establishing  a 
system  of  rewards  and  penalties  for  good  and  bad  contractor  performance  respectively. 

United  States  Army  Corp  of  Engineer’s  Construction  Contractor  Evaluation  Process 
The  USACE  comprises  approximately  34,600  civilian  and  650  military  members.  With 
its  staff  of  biologists,  engineers,  geologists,  hydrologists,  natural  resource  managers  and  other 
professionals,  the  USACE’s  mission  is  to  provide  “quality,  responsive  engineering  services  to 
the  nation  (US  Army  Corp  of  Engineers,  2000).”  This  mission  includes: 

•  Planning,  designing,  building,  and  operating  water  resources  and  other  civil  works 
projects  (Navigation,  Flood  Control,  Environmental  Protection,  Disaster  Response,  etc.) 

•  Designing  and  managing  the  construction  of  military  facilities  for  the  Army  and  Air 
Force.  (Military  Construction) 

•  Providing  design  and  construction  management  support  for  other  Defense  and  federal 
agencies.  (Interagency  and  International  Services) 

Just  as  the  USAF  contractor  selection  method  is  governed  by  the  Federal  Acquisition  Regulation 
(FAR),  so  is  the  United  States  Army  Corp  of  Engineer  (USACE).  In  accordance  with  FAR, 
subpart  36.201  Evaluation  of  contractor  performance,  “contracting  activity  shall  evaluate 
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contractor  performance  and  prepare  a  performance  report  using  the  SF  1420  (see  Appendix  B), 
Performance  Evaluation  (Construction  Contracts),  for  each  construction  contract  of  $550,000  or 
more  or  more  than  $10,000  if  the  contract  was  terminated  for  default  (Department  of  Defense, 
2005).”  Defense  Federal  Acquisition  Regulation  Supplement  (DFARS)  236.201  Evaluation  of 
Contractor  Performance,  requires  the  use  of  DD  Form  2626  (see  Appendix  A),  Performance 
Evaluation  (Construction)  instead  of  SF  1420  (Department  of  Defense,  2006). 

USACE  Regulation  ER  415-1-17,  Contractor  Performance  Evaluation,  establishes  the 
procedures  for  evaluating  construction  contractor  performance  for  all  headquarter  (HQ)  USACE 
elements  and  Major  Subordinate  Commands  (MSC)  that  are  responsible  for  military  and  civil 
construction  contracts.  Prior  to  source  selection,  USACE  contracting  officers  are  required  to 
retrieve  all  performance  evaluations  on  file  that  pertain  to  all  prospective  awardees  and  “make  a 
determination  of  responsibility  regarding  the  contractors’  previous  performance  on  DoD 
construction  contracts  (U.S.  Army  Corps  of  Engineers,  1993).”  According  to  a  USACE  resident 
engineer  only  the  contractor’s  overall  performance  score  on  previous  contracts  is  provided  to  the 
decision  team  during  the  source  selection  process.  Those  contractors  who  receive  an 
unsatisfactory  performance  evaluation  report  may  be  barred  or  excluded  from  Government 
contracting  or  Government  subcontracting  for  a  reasonable,  specified  time  period  (Department  of 
Defense,  2005).”  Conversely,  those  contractors  who  receive  an  outstanding  performance 
evaluation  report  are  considered  for  USACE  recognition  and  Division  awards. 

DD  Form  2626  is  used  for  both  interim  and  final  contractor  performance  evaluations. 
Interim  ratings  serve  as  a  valuable  tool  for  identifying  unsatisfactory  performance  from  a 
contractor.  The  interim  evaluation  provides  the  contractor  the  feedback  necessary  to  improve 
their  performance,  correct  deficiencies,  and  avoid  a  final  unsatisfactory  rating.  The  performance 
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report  is  typically  prepared  by  the  project  manager  and  then  reviewed  for  accuracy  and  fairness 
by  the  resident  engineer.  Once  the  contractor  has  had  the  opportunity  to  review  and  refute  any 
portion  of  the  DD  Form  2626,  the  form  is  sent  to  the  district  USACE  office.  It  is  the  district 
office  manager  who  is  responsible  for  approving  a  finalized  copy  of  DD  Form  2626,  which  is 
then  transmitted  to  the  USACE,  North  Pacific  Division  (NPD)  central  database  system  (CCASS). 

CCASS  stands  for  Construction  Contractor  Appraisal  Support  System;  it  is  a  web- 
enabled  application  that  supports  the  “completion,  distribution,  and  retrieval  of  construction 
contract  performance  evaluations  (DD  Form  2626)  (Naval  Sea  Logistics  Center  ,  2008).”  The 
evaluations  of  a  contractor’s  performance  provide  either  a  positive  or  negative  record  on  a  given 
contract.  “Each  evaluation  is  based  on  objective  facts  and  supported  by  contract  management 
data,  such  as  contract  performance  elements  that  evaluate  quality,  timely  performance, 
effectiveness  of  management,  and  compliance  with  contract  terms,  labor  standards,  and  safety 
requirements  (Naval  Sea  Logistics  Center  ,  2008).” 

The  method  used  by  the  USACE  for  evaluating  and  selecting  construction  contractors  has 
been  in  place  for  at  least  15  years.  If,  through  thorough  statistical  analysis,  the  data  collected  on 
DD  Form  2626  is  both  reliable  and  valid  for  evaluating  contractor  performance,  then  this  method 
can  be  adopted  by  the  USAF  in  their  effort  to  strengthen  the  USAF  rating  system  of  contractor 
performance.  It  could  be  used  to  standardize  performance  criteria  and  eliminate  inconsistencies 
between  USAF  MAJCOMs  and  bases,  and  ensure  best  value  is  achieved  for  the  government  by 
establishing  a  system  of  rewards  and  penalties  for  good  and  bad  contractor  performance 
respectively.  The  next  section  explores  various  models  that  have  been  developed  using  data, 
similar  to  the  data  collected  on  DD  Form  2626,  to  predict  contractor  performance. 
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Predicting  Contractor  Performance 

The  objective  of  this  research  is  to  evaluate  and  recommend  standardized  parameters  for 
evaluating  qualified  USAF  contractors  and  predicting  contractor  performance.  “Contractor 
selection  is  a  decisive  event  for  project  success  (Alarcon  &  Mourgues,  2002).”  It  requires 
complex  communication  between  an  array  of  construction  industry  clients  and  various 
construction  companies.  “For  that  reason,  the  success  or  failure  of  the  project  depends  on  this 
interface,  because  it  is  the  magnifying  glass  used  to  look  for  the  contractor  who  satisfies  the 
project  objectives  in  the  best  way  (Alarcon  &  Mourgues,  2002).”  When  considering  the 
weaknesses  and  limitations  of  source  selection  practices,  Holt,  et  al.,  in  their  article  A  Review  of 
Contractor  Selection  Practice  in  the  U.K.  Construction  Industry,  identified  four  major  selection 
deficiencies.  First,  and  foremost,  there  is  no  universal  approach  to  contractor  selection; 
secondly,  confidence  in  the  results  of  the  prequalification  declines  over  time;  third,  there  exists 
too  much  reliance  on  the  contractor’s  bid  price  during  final  the  stages  of  selection;  and  finally, 
there  is  an  overreliance  on  subjective  analysis  of  contractor’s  past  performance  (Holt, 
Olomolaiye,  &  Harris,  1995). 

Many  contractor  predictor  models  exist  in  the  available  literature.  In  his  article  Which 
Contractor  Selection  Model?,  Dr.  Gary  Holt  considers  seven  different  selection  models  over  five 
characteristics:  known  usage,  degree  of  subjectivity,  nature  of  input  data,  nature  of  output,  and 
future  scope  to  problem.  Table  3,  on  page  25,  summarizes  Holt’s  findings  for  each  of  the  seven 
selection  models. 

The  Bespoke  approach  to  contractor  selection  is  typically  an  ad  hoc  evaluation/selection 
method  that  is  developed  by  the  particular  contractor  source  selector.  The  initial  stages  of  the 
Bespoke  approach  includes  an  “investigation  of  contractors’  submissions  for  preliminary 
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conforming  criteria,  often  referred  to  as  ‘cut-off  points’  or  ‘musts’  (Holt,  1998).”  The 
conforming  criteria  are  usually  in  the  form  of  a  binary  decision  (YES/NO),  such  that  non- 
conforming  tenders  are  instantly  rejected.  There  exists  potential  risk  for  rejecting  a  ‘good’ 
contractor  by  employing  this  method  of  contractor  selection.  An  important  consideration  of  the 
Bespoke  approach  is  that  it  does  not  compare  contractors  relative  to  each  other;  instead  by 
employing  a  systematic  reject  method  developed  internally  by  the  construction  client’s  project 
management  team,  a  final  tender  is  selected  which  may  or  may  not  be  the  overall  “best” 
contractor.  This  approach  is  commonly  used  in  the  industry.  Though,  there  is  little  research  to 
support  that  this  is  a  practicable  method  for  selecting  “good”  contractors.  Holt  recommends  a 
numeric  measure  or  score  for  improving  the  Bespoke  Approach.  A  numeric  score  would  allow 
decisions  makers  a  means  for  comparing  contractors  relative  to  each  other. 

“Multi-attribute  analysis  considers  a  decision  alternative  with  respect  to  several  of  that 
alternative's  attributes  (Holt,  1998).”  Attributes  are  characteristics  that  can  be  measured; 
attributes  are  measured  against  objectives  -  “a  contractor  attribute  represents  one  aspect  of  a 
decision  option  with  respect  to  a  client  objective  (Holt,  1998).”  It  is  important  to  note  that  some 
attributes  may  be  qualitatively  measured  therefore  not  all  attributes  are  quantifiable.  Like  the 
Bespoke  Approach,  multi-attribute  analysis  is  commonly  used  in  industry  and  like  the  Bespoke 
Approach,  there  is  limited  literature  available  to  support  the  use  of  multi-attribute  analysis  as  a 
successful  method  for  selecting  “good”  perfonning  contractors.  Using  multi-attribute  analysis, 
Minchin  and  Smith  propose  a  quality  based  contractor  rating  model  for  qualifying  and  selecting 
contractors.  Through  a  series  of  interviews  and  questionnaires  administered  to  contractors  and 
owners,  Minchin  and  Smith  developed  the  following  three  part  contractor  rating  and 
qualification  model. 
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Table  3.  Matrix  of  methodology  characteristics  with  respect  to  the  contractor  selection  (Holt,  1998) 


Characteristics 

Methodology 

Known  Usage 

Degree  of 
subjectivity 

Nature  of  input  data 

Nature  of  output 

Future  scope  to 
problem 

Bespoke  methods 

Prolific  amongst 
industry  practitioners 

Input  &  output  very 
subjective 

Descriptive,  binary, 
linguistic  subjective 

B  inary /de  scripti  ve 

Limited  in  terms  of 
being  quantitative 

Multi-attribute 

analysis 

Simple  scoring  models 
used  by  industry,  some 
usage  by  academia 

Input  reliant  upon 
subjective  evaluation 
of  attributes 

Interval  and  ordinal 
but  of  ten  subjective 

Numeric  score  and 
hence  rank  amongst 
alternatives 

Limited  in  terms  of 
being  quantitative 

Multi-attribute 
utility  theory 

Evidence  of  academic 

usage 

Input  converts 
qualitative  data  to 
quantitative 

Raw  data  is  often 
qualitative,  utility 
achieves  interval  data 

Numeric  score  and 
hence  rank  amongst 
alternatives 

Good  if  representative 
utility  curves  are 
derived;  needs 
research 

Multiple  regression 

Evidence  of  academic 

usage 

Achieving  interval 
data  prone  to 
subjective  evaluation 

Interval  predictive 

Numeric;  further  value 

Good;  scope  for 
research 

Cluster 

analysis 

Limited 

None  if  raw 

multivariate  data  is 

used 

Multivariate 

Group  membership 
and  group 
characteristics 

Excellent;  scope  for 
future  research 

Fuzzy  set  theory 

Evidence  of  academic 

usage 

Scope  for 
development  of 
attribute  profiles 

Descriptive/qualitative 
converted  to  interval 

Group  membership 

Good;  but  may  be  too 
complex  for  easy 
acceptance  by  industry 

Multivariate 
discriminate  analysis 

Previous  usage 

Quantitative 

Multivariate 

Group 

membership/ group 
characteristics 

Previously  used  but 
broader  scope 
possible;  needs 
research 
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Part  I  includes  a  questionnaire  that  is  completed  by  the  project  manager  on  the 
contractor’s  performance.  Using  a  series  of  “yes-no”  questions  summarized  under  five 
project  managements  factors,  an  overall  project  performance  factor  from  the 
questionnaire  (PPFq)  is  determined.  The  following  equation  (equation  1)  is  used  to 
determine  a  contractor’s  overall  project  performance  factor  from  the  questionnaire  score: 

PPFq  =  .30(Project  Personnel) 

+  .20(Project  Management/Control) 

+  .20(Schedule  Adherence) 

+  .20(Contractor  Organization) 

+  .10(Plant  and  Equipment) 

“The  weights  in  equation  1  were  determined  from  input  from  the  focus  groups,  surveys, 
and  investigators’  experience  (Minchin  Jr.  &  Smith,  2005).”  Using  tests  performed  by 
project  managers  on  materials  and  workmanship  of  the  project,  an  overall  project 
performance  factor  from  data  (PPF<|)  is  determined.  Combining  the  PPFq  and  PPFd  scores 
results  in  the  overall  project  performance  factor  score.  The  follow  equation  (equation  2) 
is  used  to  determine  the  overall  project  performance  score  (PPF): 


PPF  =  0.2(PPFd)  +  0.8  (PPFq) 

The  second  part  of  the  model  is  to  determine  a  company’s  performance  rating  (CF)  using 
the  PPF  score.  A  contractor’s  performance  rating  is  a  cumulative  score  that  changes  as 
they  complete  additional  projects  over  time.  The  following  equation  (equation  3)  is  used 
to  determine  the  contractor’s  performance  rating  (CF): 


CF  - 


Y,  PPF 
N 
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“where  N  =  the  number  of  projects  completed  by  the  contractor  during  the  rating  period 
(Minchin  Jr.  &  Smith,  2005).”  The  third  part  of  the  model  uses  the  contractor’s 
performance  rating  for  qualification  and  bid  selection.  In  their  example,  Minchin  and 
Smith  calculate  a  “C”  factor  by  multiplying  CF  by  some  monetary  amount  and 
subtracting  that  amount  from  the  contractor’s  bid  amount  (Bid  Amount  -  (CF  x  $/CF)). 
Table  4  summarizes  a  hypothetical  bid  process  which  incorporates  the  “C”  factor  for 
selection. 


Table  4.  Effect  of  "C"  Factor  (Minchin  Jr.  &  Smith,  2005) 


Contractor 

Bid  Amount 

CF 

$/CF 

"C"  Factor 

Total  Bid 

A 

$2,175,000 

91 

$10,000 

$910,000 

$1,265,000 

B 

$2,200,000 

88 

$10,000 

$880,000 

$1,320,000 

C 

$2,225,000 

97 

$10,000 

$970,000 

$1,255,000 

From  the  example,  from  initial  analysis,  Contractor  A  has  submitted  the  lowest  bid. 
However,  by  incorporating  the  contractor’s  performance  rating,  Contractor  C’s  high  CF 
score  results  in  a  lower  overall  bid.  This  is  an  example  of  how  a  contractor’s  past 
performance  is  rewarded  in  future  contract  selections. 

The  next  method  Holt  addresses  is  multi-attribute  utility  analysis.  Similar  to 
multi-attribute  analysis,  multi-attribute  utility  theory  examines  many  attributes  associated 
with  various  decision  alternatives  when  making  a  decision.  Multi-attribute  utility  theory 
takes  multi-attribute  analysis  a  step  further  by  quantifying  subjective  components  through 
application  of  utility.  Utility  is  a  measure  of  desirability  or  satisfaction  with  respect  to  a 
characteristic  (attribute)  of  an  alternative  (contractor)  and  it  takes  values  from  zero  to  1.0 
(Holt,  1998).  It  is  a  means  weight  each  attribute  being  considered  for  each  alternative.  It 
allows  the  decision  maker  to  quantify  both  tangible  and  intangible  characteristics  during 


36 


the  contractor  selection  process.  Multi-attribute  utility  theory  takes  subjective  criteria 
and  objectively  scores  it.  The  application  of  multi-attribute  utility  theory  has  mostly  been 
academic  because  each  organization  that  would  apply  multi-attribute  utility  theory  in 
their  selection  processes  requires  their  own  unique  utility  curve.  Little  to  no  research 
exists  supporting  the  actual  application  of  multi-attribute  utility  theory  as  a  successful 
tool  for  selecting  “good”  performing  contractors. 

The  next  approach  Holt  describes  in  his  paper  on  contractor  selection  is  multiple 
regression.  “Multiple  regression  is  a  statistical  technique  whereby  an  equation  is 
constructed  to  observe  and  ultimately  predict  the  effect  of  several  independent  variables 
upon  a  dependent  variable  (Holt,  1998).”  The  outcome  of  multiple  regression  is  a 
numeric  value  ‘Y’  that  is  dependent  on  several  independent  variables  typically 
represented  in  a  regression  equation  as  either  ‘V’  or  ‘X’.  Through  statistical  analysis  of 
several  scenarios  a  multiple  regression  equation  is  developed.  The  equation  is  expressed 
as  follows: 

n 

Y*  =  C0YJCixi 

i= 1 

where: 

Y*  =  dependent  variable 

X;  =  independent  variables 

Ci  =  partial  regression  coefficients 

Co  =  a  constant;  the  point  on  the  y  axis  the  regression  line  crosses 

n  =  the  number  of  attributes  considered  in  the  analysis 
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R"  is  a  term  used  to  identify  the  ‘goodness  of  fit’  of  the  regression  line.  For  instance,  an 
R"  value  of  0.8  indicates  that  for  the  given  equation,  80%  of  the  movement  in  Y*  can  be 
attributed  to  movement  in  Xi.  “The  larger  the  R”  is,  then  the  more  accurate  a  predictor  is 
the  equation  based  on  the  input  data  upon  which  it  was  built  (Holt,  1998).”  One  such 
equation  for  prediction  contractor  performance  has  been  proposed  as  follows: 

Y*  =  0.311  +  0.151X!  +  0.035X8  +  0.154X9  -  0.159X19  -  0.031X20  +  0.232X21 
where: 

Y*  =  prequalification  score 

Xi  =  size  of  contractor  organization 

Xg  =  quality  of  bank  reference 

X9  =  quality  of  creditor  references 

X19  =  past  performance  (time  overruns) 

X2o  =  past  performance  (cost  overruns) 

X21  =  past  performance  (quality  achieved) 

R2  =  0.96 

Given  an  R"  of  0.96,  one  would  conclude  that  this  model  for  predicting  contractor 
perfonnance  is  highly  accurate.  Using  clients’  tender  evaluation  criteria,  Wong  developed 
a  multiple  regression  model  to  predict  contractor  performance.  Wong’s  model  was 
validated  with  75%  accuracy  using  20  independent  cases  of  contractor  performance 
(Wong,  2004).  Though  the  model  has  a  high  predictability,  it  has  not  been  field  tested  for 
accuracy  and  industrial  application. 

The  fifth  approach  to  contractor  selection  Holt  discusses  is  cluster  analysis. 

Cluster  analysis  provides  a  means  for  dividing  a  large  pool  of  potential  contractors  into 
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small,  manageable  subsets  of  similar  character.  These  small  subsets  of  characteristics  can 
then  be  analyzed  and  the  best  contractor  subset  identified  as  a  means  of  prequalification 
for  future  bid  invitation.  There  are  three  primary  benefits  of  approaching  contractor 
selection  with  cluster  analysis.  First,  using  a  limited  number  of  controlling  criteria  to  the 
entire  original  set  of  contractors  facilitates  effective  investigation  of  all  members. 

Second,  cluster  analysis  reduces  the  risk  of  rejecting  ‘good’  contractors  during  the  early 
stages  of  contractor  selection.  Third,  by  committing  more  time  resources  for  information 
gathering  during  the  selection  phase  of  construction  the  selection  body  maximizes  the 
potential  of  selecting  the  best  alternative  (contractor).  To  employ  the  cluster  analysis 
method,  the  decision  makers  describe  each  contractor  by  using  a  set  of  numerical 
attribute  scores.  These  scores  are  then  used  in  “a  classification  algorithm  to  group  the 
contractors  into  a  number  of  clusters  such  that  contractors  within  classes  are  similar  and 
unlike  those  from  other  clusters  (Holt,  1998).”  In  cluster  analysis,  the  most 
discriminating  selection  factors  are  identified  and  typically  only  significant 
discriminators  are  considered  when  deciding  between  two  alternatives.  Because  there  is 
limited  use  of  cluster  analysis  to  date  little  to  no  research  exists  supporting  the  use  of 
cluster  analysis  as  a  successful  tool  for  predicting  and  selecting  “good”  performing 
contractors. 

The  sixth  contractor  selection  model  addressed  by  Holt  is  Fuzzy  set  theory. 
Uncertainty  is  a  function  of  imprecision,  randomness,  and  ambiguity.  Probability  theory 
addresses  randomness  as  it  attempts  predict  future  events  based  on  past  events.  However, 
it  is  difficult  to  fit  contractor  selection  in  this  type  of  predictor  model.  Given  this 
difficulty,  the  Fuzzy  set  theory  is  designed  to  “model  human  judgment  and  cope  with 
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uncertainty  (Holt,  1998).”  Like  the  Bespoke  approach  to  contractor  selection,  Fuzzy  set 
theory  incorporates  ‘conforming  criteria’  to  contractor  selection.  However,  unlike  the 
Bespoke  approach,  rather  than  identifying  contractors  as  either  completely  meeting 
conforming  criteria  or  not  (binary  decision),  Fuzzy  set  theory  allows  for  partial 
membership  in  a  set  of  conforming  criteria.  The  degree  of  membership  is  measured  with 
a  membership  value  Mv  where  0  <  Mv  <1.0  and  1.0  equals  maximum  strength  of 
membership. 


Fuzzy  set  theory  was  further  developed  into  cognitive  maps  by  Manjula 
Dissanayake  and  Simann  AbouRizk  in  2007.  “A  cognitive  map  is  a  representation  of  an 
individual’s  (or  group  of  individual’s)  knowledge  of  their  spatial  environment.  Kosko 
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(1986)  developed  the  fuzzy  extension  of  the  cognitive  map,  namely  Fuzzy  cognitive 
maps,  to  represent  causal  reasoning  (Dissanayake  &  AbouRizk,  2007).”  Figure  2  is  an 
example  of  a  Fuzzy  cognitive  map  developed  by  Dissanayake  and  AbouRizk  and 
incorporates  eleven  construction  performance  concepts.  Seventeen  construction 
performance  concepts  were  originally  identified  in  their  study  and  categorized  as  either 
external,  internal,  or  goal  concepts.  According  to  Dissanayake  and  AbouRizk,  the 
construction  project  management  team  can  perform  construction  performance  concept 
identification  and  should  carry  it  out  on  a  project  by  project  basis.  Due  to  the  its 
complexity,  Fuzzy  set  theory  has  limited  use  in  the  industry  and  therefore,  little  to  no 
research  exists  supporting  the  use  of  Fuzzy  set  theory  as  a  successful  tool  for  predicting 
and  selecting  “good”  performing  contractors. 

The  final  contractor  selection  method  discussed  by  Holt  is  multivariate 
discriminate  analysis.  “Multivariate  discriminate  analysis  studies  the  differences  between 
two  or  more  objects  with  respect  to  several  variables,  simultaneously  (Holt,  1998).”  By 
examining  several  variables  such  as  contractors  past  performance,  decision  makers  can 
identify  which  attributes  are  discriminators,  how  the  attributes  can  then  be  used  in  an 
algorithm  to  predict  performance,  and  the  accuracy  of  the  derived  equation.  Multivariate 
discriminate  analysis  is  useful  in  identifying  the  most  powerful  discriminators  between 
contractors.  Multivariate  discriminate  analysis  begins  with  the  identification  of  the  most 
discriminating  variable.  By  combining  this  variable  with  each  of  the  other  variables,  the 
next  most  discriminating  factor  is  identified.  This  process  is  continued  until  “very  little 
discrimination  is  gained  by  inclusion  of  any  further  variable  (Holt,  1998).”  This  method 
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was  applied  to  predict  future  performance  of  Hong  Kong  contractors.  The  resulting 
discriminating  function  is  as  follows: 

%t  =  Q>  +  c1x1  +  c2x2  +  — i-  c6x6 

where: 

Zt  =  predictive  contractor  performance  index 

Co  =  constant 

Ci -6  =  discriminating  coefficients 

Xi  =  complexity  of  project 

X2  =  percentage  of  professionally  qualified  staff 
X3  =  project  leaders  experience 
X4  =  contractor’s  past  performance  or  image 
X5  =  origin  of  the  company 

X6  =  construction  owner’s  control 

In  their  PhD  thesis,  A  Methodology  for  Predicting  Company  Failure  in  the 
Construction  Industry ,  Adnan  Fadhil  Abidali  and  Frank  Harris  developed  a  Z-score 
model  for  vetting  construction  companies  during  source  selection  prequalification.  The 
Z-score  model  was  developed  to  predict  construction  contractor  solvency.  Ratio  models 
were  developed  for  application  in  construction  in  order  to  “minimize  risk  for  client 
organizations  and  corporate  lending  institutions  that  usually  have  a  direct  business 
relationship  with  construction  companies  (Edum-Fotwe,  Price,  &  Thorpe,  1996).”  Such 
ratio  models  can  provide  early  warning  to  construction  company  clients,  thus  serving  as  a 
monitoring  tool  for  avoiding  poor  corporate  performance  or  possible  insolvency. 
Insolvency  is  used  as  a  broader  term  which  includes  liquidation,  receivership,  and 
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administration  of  a  company  by  bankers  or  others  with  a  financial  stake  (Langford, 
Iyagba,  &  Komba,  1993).  The  seven-variable  model  developed  by  Abidali  and  Harris 
produces  a  Z-score  that  can  be  used  to  predict  a  construction  company’s  long  tenn 
solvency.  The  model  is  expressed  as  follows: 

Z  =  14.6  +  82.0*!  -  14.5Z2  +  2.5X3  -  1.2X4  +  3.55XS  -  3.55X6  -  3.0X7 
In  this  model  Xi  represents  the  ratio  of  profit  after  tax  and  interest  to  net  capital 
employed.  “This  is  a  profitability  measure  and  takes  into  account  all  the  net  assets  plus 
the  short-term  loans  used  to  finance  the  company.  This  ratio  is  a  valuable  guide  to  the 
profitability  of  companies.  The  value  appears  positive  in  solvent  companies  and  tends 
towards  the  negative  in  failed  companies  (Abidali  &  Harris,  1995).”  X2  represents  the 
ratio  of  current  assets  to  net  assets  and  measures  the  financial  leverage  of  a  construction 
company.  Firms  that  have  failed  consistently  have  fewer  current  assets  than  non-failed 
firms.  “The  ability  of  a  firm  to  meet  its  short-term  financial  obligations  without  having  to 
liquidate  its  long-term  assets  is  an  important  factor  in  the  consideration  of  lenders;  the 
extreme  case  of  such  an  inability  is  bankruptcy  (Abidali  &  Harris,  1995).”  X3  represents 
the  ratio  of  turnover  to  net  assets;  it’s  a  measure  a  company’s  efficient  use  of  its 
productive  capacity.  X4  represents  the  ratio  of  short  term  loans  to  profits  before  tax  and 
interest  and  measures  the  company’s  liquidity.  X5  represents  the  tax  trend  over  three 
years.  “The  tax  trend  tends  toward  the  negative  in  failed  companies.  As  a  company 
becomes  “better  off’  the  trend  increases  (Abidali  &  Harris,  1995).”  X6  represents  the 
profit  after  tax  trend  over  three  years.  “The  earnings  after  tax  trend  towards  the  negative 
in  failed  companies.  Again,  as  a  company  becomes  better  off  the  trend  increases; 
whereas  failed  companies’  tax  trend  decreases  (Abidali  &  Harris,  1995).”  X7  represents 
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the  short  term  loan  trend  over  three  years  and  measures  the  liquidity  over  several  years. 
“Generally,  failed  companies  are  highly  dependent  on  short-term  loans  more  than  non- 
failed  firms.  As  a  company  becomes  worse  off  the  trend  increases,  reaching  crisis  level 
before  collapse  (Abidali  &  Harris,  1995).”  According  to  Abidali  and  Harris  a  Z-score  of 
2.94  or  greater  predicts  long  term  solvency  for  a  construction  company. 

When  the  Z-score  model  was  tested,  90%  of  the  firms  were  correctly  classified 
into  the  non-failed  group  and  100%  of  the  failed  companies  were  correctly  classified.  On 
its  own,  the  Z-score  cannot  predict  failure;  it  merely  provides  a  snapshot  of  a  company’s 
financial  solvency  under  the  current  management  and  compares  it  to  similar  companies. 
Instead  the  model  indicates  the  likelihood  of  failure;  non-financial  analysis  for  companies 
is  necessary  to  reinforce  the  predictive  capacity  of  the  model.  “There  can  be  little  doubt 
about  the  usefulness  of  financial  ratios  as  a  management  evaluation  tool  for  the 
construction  industry.  They  serve  as  early  warning  systems  by  indicating  whether  an 
organization  is  in  good  financial  standing  or  exhibits  characteristics  of  already  failed 
companies  (Edum-Fotwe,  Price,  &  Thorpe,  1996).”  More  specifically,  the  ratios  are 
“indicators  of  past  events  and  the  trend  may  continue  if  managers  of  the  company  do 
nothing  to  change  the  situation  (Langford,  Iyagba,  &  Komba,  1993).”  The  information 
required  to  use  multivariate  discriminant  analysis,  such  as  a  Z-score  model  for  predicting 
and  selecting  “good”  contractors,  is  detailed  and  not  easily  attained  because  the 
contractor  must  provide  it.  This  makes  industrial  application  limited  to  non-existent. 
Therefore,  little  to  no  research  exists  supporting  the  use  of  multivariate  discriminant 
analysis  as  a  successful  tool  for  predicting  and  selecting  “good”  performing  contractors. 

This  research  will  use  statistical  analysis  to  test  the  reliability  of  DD  Form  2626 


44 


and  then  apply  multiple  regression  to  develop  and  validate  a  contractor  performance 


predictor  model  using  the  DD  Form  2626  data.  As  noted  by  Holt,  multiple  regression 
shows  evidence  of  academic  usage,  the  nature  of  the  input  data  is  predictive,  and  the 
nature  of  the  output  is  numeric  or  quantitative.  As  discussed  in  Chapter  1,  the  current 
method  used  to  identify  the  USAF’s  overall  confidence  in  a  contractor’s  future 
performance  is  a  subjective  rating  system  (Wright,  1999).  A  rigorous  system  for  rating  or 
determining  a  contractor’s  future  performance  does  not  exist  for  the  USAF.  If  a  numeric 
value  can  be  determined  for  predicting  contractor  performance  using  multiple  regression 
on  the  data  from  DD  Form  2626,  then  the  USAF  could  have  a  more  rigorous  system  for 
rating  and  predicting  a  contractor’s  performance.  The  next  section  of  this  chapter  will 
discuss  the  performance  metrics  used  in  the  statistical  analysis  of  data  supplied  by  DD 
Form  2626. 

Study  Performance  Metrics 

This  research  will  compare  a  contractor’s  overall  contractor  performance  rating  to 
the  individual  performance  item  ratings  and  the  contractor’s  overall  schedule. 
Construction  researchers  and  practitioners  have  paid  close  attention  to  contractor 
evaluation  methods.  Most  of  the  research  conducted  on  contractor  evaluations  indicates 
that  the  evaluation  criterion  has  remained  unchanged  over  the  years.  Researchers  known 
in  the  field  of  contractor  evaluation  criteria  studies  generally  study  a  contractor’s 
performance  as  it  relates  to  financial,  managerial,  technical,  health  and  safety,  quality, 
and  past  performance  aspects  (Wong,  Holt,  &  Cooper,  2000).  Therefore,  it  is  not 
surprising  that  the  contractor  performance  evaluation  tool  used  by  the  USACE,  DD  Form 
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2626,  is  divided  into  five  performance  elements.  The  five  performance  elements  are 
quality  control,  effectiveness  of  management,  timely  performance,  compliance  with  labor 
standards,  and  compliance  with  safety  standards.  The  contractor’s  overall  performance 
rating,  which  is  also  provided  on  DD  Form  2626,  should  reflect  their  ratings  in  each  of 
the  five  performance  elements.  This  method  of  using  performance  elements  (or  facets)  to 
measure  overall  performance  is  the  same  method  used  by  Judge,  et  al.  when  measuring 
overall  job  satisfaction  to  identify  relationships  between  job  satisfaction  and  overall  job 
performance.  In  their  study,  Judge,  et  al.  measured  overall  job  satisfaction  by  measuring 
specific  facets  of  the  job  situation  to  include  supervision,  coworkers,  opportunity  for 
advancement.  These  facets  were  combined  to  form  a  measure  of  overall  job  satisfaction 
(Judge,  Thoresen,  Bono,  &  Patton,  2001). 

DD  Form  2626  consists  of  an  overall  contractor  performance  rating  and  33 
performance  items  that  are  each  assigned  to  one  of  five  performance  elements; 
performance  items  are  considered  a  sub-measure  of  performance  elements.  Each  of  the 
33  performance  items  are  rated  as  either  outstanding,  above  average,  satisfactory, 
marginal,  or  unsatisfactory.  The  first  performance  element.  Quality  Control,  is  measured 
by  the  following  eleven  performance  items:  Quality  of  Workmanship,  Adequacy  of  the 
Construction  Quality  Control  Plan,  Implementation  of  the  Construction  Quality  Control 
Plan,  Quality  of  the  Quality  Control  Documentation,  Storage  of  Materials,  Adequacy  of 
Submittals,  Adequacy  of  Quality  Control  Testing,  Adequacy  of  As-Builts,  Use  of 
Specified  Materials,  and  Identification/Correction  of  Deficient  Work  in  a  Timely  Manner. 
According  to  USACE  Regulation  ER  415-1-17  Contractor  Performance  Evaluations, 
“Quality  of  Work  reflects  the  contractor's  management  of  the  quality  control  program,  as 
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well  as  the  quality  of  the  work  which  is  placed  (U.S.  Army  Corps  of  Engineers,  1993).” 
The  USACE  is  not  the  only  government  organization  that  emphasizes  quality  control  as  a 
sub-measure  of  a  contractor’s  overall  performance.  Utah’s  Department  of  Transportation 
uses  a  standardized  form  for  evaluating  construction  contractor’s  performance.  Quality 
control  is  measured  by  five  performance  items  to  include  adequacy  of  materials.  Ten 
percent  of  the  contractor’s  overall  performance  rating  is  based  on  quality  control.  In 
addition,  the  City  of  Los  Angeles  evaluates  a  construction  contractor’s  performance  on 
“organization,  procedures,  competence  of  personnel,  and  effectiveness  of  the  contractor’s 
quality  control  on  the  project  (City  of  Los  Angeles,  2004).” 

The  second  performance  element,  Effectiveness  of  Management,  is  measured  by 
the  following  nine  performance  items:  Cooperation  and  Responsiveness,  Management  of 
Resources  and  Personnel,  Coordination  and  Control  of  Subcontractor(s),  Adequacy  of 
Site  Clean-Up,  Effectiveness  of  Job-Site  Supervision,  Compliance  with  Laws  and 
Regulations,  Professional  Conduct,  Review  and  Resolution  of  Subcontractor’s  Issues,  and 
Implementation  of  Subcontracting  Plan.  Research  has  shown  that  site  management  is  a 
crucial  factor  to  a  successful  project  outcome  (Holt,  Olomolaiye,  &  Harris,  1994).  In 
addition  to  quality  control,  Utah’s  Department  of  Transportation  considers  effectiveness 
of  management  when  rating  a  contractor’s  performance.  They  value  adequate  support  of 
subcontractors,  resolution  of  delays  quickly  and  efficiently,  and  project  supervision  that 
results  in  a  positive  impact  on  the  project. 

The  third  performance  element,  Timely  Performance,  is  measured  the  following 
seven  performance  items:  Adequacy  of  Initial  Progress  Schedule,  Adherence  to 
Approved  Schedule,  Resolution  of  Delays,  Submission  of  Required  Documentation, 
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Completion  of  Punchlist  Items,  Submission  of  Updated  and  Revised  Progress  Schedules, 
and  Warranty  Response.  In  the  state  of  Connecticut,  annual  performance  ratings  are 
conducted  on  all  contractors  and  subcontractors.  One  of  the  five  elements  in 
Connecticut’s  performance  questionnaire  is  adherence  to  project  schedule.  The  Utah 
Department  of  Transportation  (UDOT)  also  rates  timely  performance  as  an  important 
element  when  evaluating  a  contractor’s  performance  (Minchin  Jr.  &  Smith,  2001).  In  an 
article  dated  June  1999,  researchers  Chee  H.  Wong,  Gary  D.  Holt,  and  Patricia  A.  Cooper 
surveyed  construction  clients’  preferred  contractor  attributes  used  in  the  tender  selection 
process.  The  survey  required  respondents  to  rank  order  37  various  contractor 
performance  criteria.  Of  the  86  completed  surveys,  the  ability  for  a  contractor  to 
complete  a  project  on  time  was  consistently  ranked  as  either  a  number  one  or  number  two 
priority  in  a  list  of  37  contractor  performance  criteria  (Wong,  Holt,  &  Cooper,  2000). 

The  fourth  performance  element,  Compliance  with  Labor  Standards,  is  measured 
by  the  following  three  performance  items:  Correction  of  Noted  Deficiencies,  Payrolls 
Properly  Completed  and  Submitted,  and  Compliance  with  Specific  Attention  to  the 
Davis-Bacon  Act  and  Equal  Employment  Opportunity  (EEO)  Requirements.  In  the  state 
of  West  Virginia,  before  a  contractor  can  qualify  for  work,  they  must  first  obtain  a  license 
from  the  Secretary  of  State.  There  are  five  criteria  that  must  be  met  in  order  to  obtain  a 
license,  one  of  which  is  adherence  to  Davis-Bacon  pay  scales.  The  Utah  Department  of 
Transportation  rates  a  contractor’s  compliance  with  EEO  as  part  of  the  performance 
evaluation  (Minchin  Jr.  &  Smith,  2001).  In  addition,  the  City  of  Los  Angeles  construction 
contractor’s  performance  evaluation  includes  compliance  with  labor  standards. 

The  fifth  performance  element,  Compliance  with  Safety  Standards,  is  measured 
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by  the  following  three  performance  items:  Adequacy  of  Safety  Plan,  Implementation  of 
Safety  Plan,  and  Correction  of  Noted  Deficiencies.  According  to  a  report  published  by  R. 
Edward  Minchin  Jr.  and  Gary  R.  Smith  titled,  Quality-Based  Performance  Rating  of 
Contractors  for  Prequalification  and  Bidding  Purposes,  “safety  is  an  integral  part  of 
project  quality  (Minchin  Jr.  &  Smith,  2001).”  Evaluation  of  safety  is  not  unique  to  the 
USACE’s  contractor  performance  measurement  tool  -  DD  Form  2626.  The  Missouri 
Highways  and  Transportation  Commission  (MHTC)  use  a  questionnaire  for  evaluating 
construction  contractor  performance.  The  questionnaire  is  broken  down  into  four 
categories:  quality,  prosecution  and  progress,  contract  compliance,  and  safety.  In  a 
contractor’s  overall  performance  score,  safety  accounts  for  20%  of  the  total  score.  The 
Virginia  Department  of  Transportation  (VDOT)  uses  a  performance  questionnaire  to 
determine  a  contractor’s  bidding  capacity.  The  final  report  is  divided  into  four 
categories:  prosecution  of  work,  project  communication,  safety,  and  environmental 
(Minchin  Jr.  &  Smith,  2001).  In  addition,  the  City  of  Los  Angeles  construction 
contractor’s  performance  evaluation  includes  compliance  with  safety  standards. 

While  some  information  exists  in  the  literature  to  support  the  current 
categorization  of  performance  items  as  they  appear  on  the  DD  Form  2626,  the 
information  is  limited  and  fails  to  give  construct  validity  to  the  performance  elements. 
Therefore,  face  validity  will  serve  to  support  maintaining  each  performance  item  with  its 
respective  performance  element  when  performing  statistical  analysis  on  the  research  data. 

Traditionally,  a  contractor’s  overall  project  performance  is  based  on  quality, 
schedule,  and  cost  (Ling  &  Liu,  2004).  Quality  refers  to  the  performance  of  the  product 
that  is  delivered  by  the  contractor.  In  the  construction  industry,  quality  refers  to  the 
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contractor’s  ability  to  deliver  a  civil  works  project  as  specified  by  the  customer.  Quality, 
as  a  performance  measure,  is  not  captured  on  DD  Form  2626  and  therefore,  will  not  be 
included  in  this  research.  In  addition,  DD  Form  2626  does  not  provide  enough  data  to 
perform  a  cost  performance  analysis  for  each  project.  Therefore,  this  research  will 
specifically  focus  on  schedule  with  respect  to  traditional  measures  of  project  success. 
Schedule  refers  to  a  contractor’s  ability  to  conform  to  the  agreed  upon  project  delivery 
timeline.  Project  schedule  growth  is  a  measure  of  a  contractor’s  overall  schedule 
performance  (Lee,  Thomas,  Mackens,  Chapman,  Tucker,  &  Kim,  2005).  The  equation 
for  determining  project  schedule  growth  is: 

Actual  Total  Project  Duration  -  Initial  Predicted  Project  Duration 
Initial  Predicted  Project  Duration 

For  government  contracts  it  is  very  important  that  the  contractor  meets  or  exceeds 
schedule  performance.  Generally,  government  construction  projects  support  mission 
requirements;  delays  in  construction  completion  can  have  adverse  affects  in  the 
government’s  ability  to  execute  mission  requirements.  Fonner  Chief  of  Staff  of  the  Air 
Force,  General  T.  Michael  Mosley,  echoed  this  sentiment  when  addressing  members  of 
the  House  Appropriations  Committee  Subcommittee  on  Military  Construction  and 
Veterans  Affairs.  He  informed  the  members  that  “sound  investment  in  our  installations 
postures  the  Air  Force  to  support  our  priorities  of  winning  the  global  war  on  terror, 
developing  and  caring  for  our  Airmen  and  their  families,  and  recapitalizing  and 
modernizing  our  force  (Buzanowski,  2007).”  The  final  portion  of  the  data  analysis  will 
determine  if  a  relationship  exists  between  the  contractor’s  overall  performance  rating  and 
the  overall  project  schedule  performance. 
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Summary 

The  first  sections  of  this  chapter  discussed  best  value  source  selection  and  the 
current  selection  methods  used  by  the  USAF  for  contractor  selection.  Best  value  source 
selection  is  the  framework  that  governs  contractor  selection  for  all  United  States 
government  organizations,  to  include  the  USAF.  By  studying  the  current  USAF  source 
selection  practices,  the  research  identified  the  limitations  of  the  selection  methods, 
suggesting  the  need  for  a  more  rigorous  USAF  contractor  performance  rating  system  that 
can  be  used  to  predict  future  contractor  performance.  This  is  one  of  the  transformation 
initiative  items  requested  by  the  USAF  Civil  Engineer,  General  Del  Eulberg.  Rather 
than  inventing  a  new  method  for  evaluating  and  predicting  contractor  performance,  this 
research  examined  the  current  contractor  evaluation  and  selection  methods  employed  by 
the  USACE. 

The  next  section  of  this  chapter  then  discussed  in  detail  the  various  existing 
contractor  performance  predictor  models.  Through  this  research,  the  methodology  for 
analyzing  the  USACE’s  contractor  evaluation  form,  DD  Form  2626,  for  reliability  and 
validity  was  discovered.  By  employing  statistical  analysis  this  research  will  attempt  to 
validate  DD  Form  2626  as  a  reliable  tool  for  evaluating  contractors.  If  the  evaluation 
data — specifically  the  performance  items  provided  on  DD  Form  2626  are  determined  to 
be  valid — then  through  the  use  of  linear  regression,  a  model  for  predicting  contractor 
performance  will  be  developed  and  tested  for  reliability  and  validity. 

Before  testing  the  contractor  performance  elements  for  reliability  and  validity,  an 
investigation  into  previous  research  was  conducted  in  order  to  determine  if  the 
performance  elements  identified  by  DD  Form  2626  were  supported  as  reliable  and  valid 
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tools  for  evaluating  contractor  performance.  The  last  section  of  this  chapter,  study 
performance  metrics,  discussed  the  results  of  this  investigation  and  supports  the  use  of 
the  performance  elements  provided  by  DD  Form  2626  as  appropriate  contractor 
evaluation  tools. 

Identifying  methods  for  accurately  predicting  contractor  performance  is  a  topic  of 
research  that  has  been  around  for  decades.  While  many  predictor  models  and  tools  exist 
for  selecting  a  “good”  contractor,  many  of  these  models  have  not  been  field  tested — their 
reliability  and  validity  are  non-existent.  Therefore,  use  of  any  contractor  predictor  model 
is  limited  to  academic  application  and  research.  Chapter  3  will  discuss  in  detail  the 
methodology  used  in  this  research.  Through  the  use  of  statistical  analysis  and  logistical 
regression,  this  research  will  attempt  to  validate  the  method  used  by  the  USACE  for 
contractor  performance  evaluation. 
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III.  Methodology 


Chapter  Overview 

This  chapter  describes  the  methodology  used  to  analyze  the  construction 
contractor  evaluation  method  employed  by  the  US  Army  Corp  of  Engineers  (USACE), 
specifically  DD  Form  2626.  The  procedures  used  by  this  study  are  organized  into  three 
sections:  data  source,  data  collection,  and  data  analysis.  Each  section  will  explain  the 
definitions,  decisions,  and  criteria  used  for  the  study  data  analysis. 

Data  Source 

The  first  step  of  data  analysis  was  to  identify  a  data  source  that  contained 
consistent  and  representative  project  information  for  construction  projects.  This  research 
analyzed  the  data  provided  on  DD  Form  2626  Construction  Contractor  Performance 
Evaluation.  While  not  all  of  the  construction  projects  included  in  the  data  analysis  were 
military  specific,  they  were  all  managed  and  evaluated  by  the  USACE  -  an  organization 
that  is  a  major  command  under  the  US  Army.  The  data  collected  on  each  project  is  more 
standardized  than  data  that  might  be  available  through  the  public  or  private  sector 
(Pocock,  1996).  All  finalized  and  approved  copies  of  DD  Form  2626  are  transmitted  to 
the  USACE,  North  Pacific  Division  (NPD)  central  database  system  (CCASS). 

CCASS  stands  for  Construction  Contractor  Appraisal  Support  System  and  is  a 
centralized,  web-enabled,  database  for  managing  construction  contract  performance 
evaluations.  It  is  a  subset  of  CPARS,  the  Contractor  Performance  Assessment  Reporting 
System,  which  is  a  collection  of  Past  Performance  Information  on  all  contractors  that 


53 


have  been  employed  by  the  Department  of  Defense  (DoD).  A  series  of  “checks  and 
balances”  are  employed  to  ensure  the  accuracy  of  the  data  provided  in  CCASS. 
Government  officials  are  responsible  for  evaluating  and  rating  a  contractor’s  performance 
on  a  given  project  for  a  specified  period  of  time.  The  contractor  then  has  the  opportunity 
to  review  the  evaluation  and  provide  comments  “regarding  the  Government’s  assessment 
and  to  indicate  concurrence  or  non-concurrence  with  the  overall  evaluation  (Naval  Sea 
Logistics  Center  ,  2008).”  Finally,  if  there  is  a  disagreement  between  the  Government 
and  the  Contractor,  a  senior  official  reviews  the  report  to  ensure  that  it  reflects  a  fair 
evaluation.  CCASS  was  used  as  the  data  source  to  retrieve  construction  contractor 
performance  evaluation  information  for  this  study. 

Data  Collection 

The  USACE  Construction  Division,  Louisville  Division  was  instrumental  in  the 
collection  of  project  information  from  CCASS.  Contractor  Performance  Evaluations 
were  downloaded  in  portable  document  format  (PDF)  for  all  construction  projects  that 
were  executed  in  the  Louisville  Division  area  of  jurisdiction.  All  projects  were  awarded 
between  Fiscal  Year  (FY)  2000  and  2008.  Only  finalized  and  approved  evaluation  report 
were  used  in  this  data  analysis.  A  total  of  215  construction  project  evaluation  reports 
were  downloaded  and  manually  transcribed  into  an  excel  spreadsheet.  Columns  were 
added  into  the  excel  spreadsheet  to  compute  overall  schedule  performance  for  each 
project  as  detailed  in  chapter  2  and  the  data  analysis  section  of  this  chapter.  Finally,  in 
order  to  test  the  contractor  performance  predictor  model,  finalized  construction  contractor 
evaluation  reports  must  be  available  for  model  validation.  The  215  construction  project 
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evaluation  reports  were  randomly  divided  using  the  random  number  generator  function  in 
excel.  A  total  of  50  reports  were  set  aside  for  model  validation  (Appendix  C),  the  other 
165  reports  were  used  in  the  data  analysis  (Appendix  D).  This  procedure  models  the 
linear  regression  procedure  Wong  used  when  developing  his  contractor  performance 
model.  His  model  was  derived  from  48  construction  projects  and  tested  by  20 
independent  cases  using  the  logistical  regression  technique  (Wong,  2004). 

Data  Analysis 

Performance  Metrics 

The  literature  identified  performance  metrics  used  by  USACE  construction 
project  managers  and  previous  studies  to  evaluation  and  rate  construction  contractor 
performance.  Thirty-five  performance  metrics  were  used  by  this  study  and  include: 
overall  contractor  performance  rating,  overall  project  schedule  performance,  and  thirty- 
three  performance  items.  This  method  of  using  performance  elements  (or  facets)  to 
measure  overall  performance  is  the  same  method  used  by  Judge,  et  al.  when  measuring 
overall  job  satisfaction  to  identify  relationships  between  job  satisfaction  and  overall  job 
performance.  In  their  study,  Judge,  et  al.  measured  overall  job  satisfaction  by  measuring 
specific  facets  of  the  job  situation  to  include  supervision,  co workers,  opportunity  for 
advancement.  These  facets  were  combined  to  form  a  measure  of  overall  job  satisfaction 
(Judge,  Thoresen,  Bono,  &  Patton,  2001). 

Performance  Elements  and  Performance  Items 

DD  Form  2626  is  divided  into  five  performance  elements.  The  five  performance 
elements  are  quality  control,  effectiveness  of  management,  timely  performance, 
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compliance  with  labor  standards,  and  compliance  with  safety  standards.  The  contractor’s 
overall  performance  rating,  which  is  also  provided  on  DD  Form  2626,  should  reflect  their 
ratings  in  each  of  the  five  performance  elements. 

Each  performance  element  is  supported  by  a  various  number  of  performance 
items.  Each  performance  item  is  rated  as  either  outstanding,  above  average,  satisfactory, 
marginal,  or  unsatisfactory.  The  first  performance  element  is  quality  control  and  it  is 
divided  into  the  following  eleven  performance  items:  quality  of  workmanship,  adequacy 
of  the  construction  quality  control  plan,  implementation  of  the  construction  quality 
control  plan,  quality  of  the  quality  control  documentation,  storage  of  materials,  adequacy 
of  submittals,  adequacy  of  quality  control  testing,  adequacy  of  as-builts,  use  of  specified 
materials,  and  identification  and  correction  of  deficient  work  in  a  timely  manner.  The 
second  performance  element  is  effectiveness  of  management  and  it  is  divided  into  the 
following  nine  performance  items:  cooperation  and  responsiveness,  management  of 
resources  and  personnel,  coordination  and  control  of  subcontractor(s),  adequacy  of  site 
clean-up,  effectiveness  of  job-site  supervision,  compliance  with  laws  and  regulations, 
professional  conduct,  review  and  resolution  of  subcontractor’s  issues,  and 
implementation  of  subcontracting  plan.  The  third  performance  element  is  timely 
performance  and  it  is  divided  into  the  following  seven  performance  items:  adequacy  of 
initial  progress  schedule,  adherence  to  approved  schedule,  resolution  of  delays, 
submission  of  required  documentation,  completion  of  punchlist  items,  submission  of 
updated  and  revised  progress  schedules,  and  warranty  response.  The  fourth  performance 
element  is  compliance  with  labor  standards  and  it  is  divided  into  the  following  three 
performance  items:  correction  of  noted  deficiencies,  payrolls  properly  completed  and 
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submitted,  and  compliance  with  specific  attention  to  the  Davis-Bacon  Act  and  equal 
employment  opportunity  (EEO)  requirements.  The  fifth  performance  element  is 
compliance  with  safety  standards  and  it  is  divided  into  the  following  performance  items: 
adequacy  of  safety  plan,  implementation  of  safety  plan,  and  correction  of  noted 
deficiencies.  According  to  the  resident  engineer  at  a  local  USACE  construction  office, 
the  contractor’s  overall  performance  rating  should  be  a  reflection  of  the  ratings  he  or  she 
received  in  each  of  the  various  performance  elements. 

Contractor  Overall  Performance  Rating 

Traditionally,  a  contractor’s  overall  project  performance  is  based  on  quality, 
schedule,  and  cost  (Ling  &  Liu,  2004).  Quality,  as  a  performance  measure,  is  not 
captured  on  DD  Form  2626  and  therefore,  will  not  be  included  in  this  research.  In 
addition,  DD  Form  2626  does  not  provide  enough  data  to  perform  a  cost  performance 
analysis  for  each  project.  Therefore,  this  research  will  specifically  focus  on  schedule 
with  respect  to  tradition  measures  of  project  success.  The  overall  contractor  performance 
rating  is  indicated  on  the  first  page  of  the  DD  Form  2626.  The  contractor’s  performance 
rating  is  measured  on  a  five-point  scale  using  the  following  descriptors:  outstanding, 
above  average,  satisfactory,  marginal,  and  unsatisfactory. 

Overall  Project  Schedule  Performance 

Schedule  refers  to  a  contractor’s  ability  to  conform  to  the  agreed  upon  project 
delivery  timeline.  Project  schedule  growth  is  a  measure  of  a  contractor’s  overall  schedule 
performance  (Lee,  Thomas,  Mackens,  Chapman,  Tucker,  &  Kim,  2005).  The  equation 
used  for  determining  project  schedule  growth  is: 
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Actual  Total  Project  Duration  -  Initial  Predicted  Project  Duration 
Initial  Predicted  Project  Duration 


Statistical  Analysis 

This  research  used  multivariate  analysis  to  validate  the  use  of  DD  Form  2626  for 
evaluating  and  rating  construction  contractor  performance.  Multivariate  analysis  refers 
“to  all  statistical  techniques  that  simultaneously  analyze  multiple  measurements  on 
individuals  or  objects  under  investigation  (Hair,  Black,  Babin,  Anderson,  &  Tatham, 
2006).”  The  ultimate  goal  of  this  research  was  to  identify  a  relationship  with  the  variate 
value,  or  the  Overall  Contractor  Performance  Rating,  and  the  observed  variables,  or  the 
performance  items’  measurements.  In  addition,  this  researched  identified  a  relationship 
with  the  Overall  Contractor  Performance  Rating  and  the  Overall  Cost  and  Overall 
Schedule  Performance. 

The  variate  is  the  building  block  of  multivariate  analysis.  It  represents  a  single 
value  resulting  from  a  combination  of  observed  variables.  The  variate  value  can  be  stated 
mathematically  as: 

Variate  value  =  W/X/  +  w2X2  +  w<X2  +  . . .+  wnXn 
where  Xn  is  the  observed  variable  and  wn  is  the  weight  determined  by  the  multivariate 
technique  (Hair,  Black,  Babin,  Anderson,  &  Tatham,  2006).  By  design,  the  DD  Form 
2626  implies  that  quality  control,  timely  performance,  effectiveness  of  management, 
compliance  with  labor  standards,  and  compliance  with  safety  standards  are  all  indicators 
of  a  construction  contractor’s  overall  performance.  The  purpose  of  this  research  was  to 
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determine  if  in  fact  these  five  performance  elements  and  associated  33  performance  items 
are  both  reliable  and  valid  indicators  of  a  contractor’s  overall  performance. 

In  this  research  the  variate,  or  the  Overall  Contractor  Performance  Rating,  and  the 
observed  variable,  or  Performance  Sub-elements,  were  measured  on  an  ordinal  scale 
where  the  construction  contractor’s  performance  is  rated  as  ‘outstanding’,  ‘above 
average’,  ‘satisfactory’,  ‘marginal’,  or  ‘unsatisfactory’.  For  the  purpose  of  data  analysis, 
each  rating  was  represented  numerically  where  ‘5’  =  ‘outstanding’,  ‘4’  =  ‘above 
average’,  ‘3’  =  ‘satisfactory’,  ‘2’  =  ‘marginal’,  and  ‘1’  =  ‘unsatisfactory’. 

Measurement  Error 

Measurement  error  is  “the  degree  to  which  the  observed  values  are  not 
representative  of  the  “true”  values  (Hair,  Black,  Babin,  Anderson,  &  Tatham,  2006).” 
Sources  for  measurement  error  in  this  research  include  data  entry  errors  that  occurred 
when  manually  transferring  data  from  the  contractor  performance  evaluation  form  to  the 
excel  spreadsheet.  The  use  of  an  imprecise  measurement  tool  may  also  be  a  source  of 
error;  specifically,  imposing  a  5 -point  rating  scale  for  performance  measurement  when 
maybe  some  categories  of  performance  can  only  be  accurately  measured  by  a  3-point 
rating  scale  can  cause  errors  in  the  data.  “Thus,  all  variables  used  in  multivariate 
techniques  must  be  assumed  to  have  some  degree  of  measurement  error  (Hair,  Black, 
Babin,  Anderson,  &  Tatham,  2006).”  Measurement  errors  were  considered  in  this 
research  as  they  can  mask  the  “true”  effects,  resulting  in  weakened  correlations  and  less 
precise  means. 
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Statistical  Analysis  Basic  Assumptions 

Before  statistical  analysis  was  conducted  on  the  contractor  performance  data,  a 
few  statistical  assumptions  had  to  be  met.  The  statistical  analysis  that  was  conducted  in 
this  research  is  generally  classified  as  parametric  tests.  Using  a  parametric  test  when  the 
data  does  not  meet  four  basic  assumptions  can  cause  the  results  to  be  inaccurate.  Before 
any  parametric  test  was  conducted  on  the  research  data  in  this  study,  the  assumptions 
were  checked.  According  to  Andy  Field,  author  of  Discovering  Statistics  Using  SPSS, 
for  standard  parametric  tests,  the  following  assumptions  must  be  met: 

Assumption  1:  Normally  Distributed  Data.  Normally  distributed  data  is  “a 
probability  distribution  of  a  random  variable  that  is  known  to  have  certain  properties;  it  is 
perfectly  symmetrical  (has  a  skew  of  0),  and  has  a  kurtosis  of  0  (Field,  2005).”  When 
plotted  on  a  two-dimensional  graph,  where  the  horizontal  axis  represents  all  possible 
values  of  the  variable  and  the  vertical  axis  represents  the  probability  of  those  values 
occurring,  data  that  is  normally  distributed  will  be  clustered  around  the  mean  in  a 
symmetrical,  unimodal  pattern.  This  pattern  is  commonly  referred  to  as  a  bell-shaped 
curve  or  normal  curve  (Hair,  Black,  Babin,  Anderson,  &  Tatham,  2006).  Since  the 
research  conducted  in  this  study  involves  more  than  two  variables,  multivariate  normality 
must  be  considered.  “Multivariate  normality  (the  combination  of  two  or  more  variables) 
means  that  the  individual  variables  are  normal  in  a  univariate  sense  and  that  their 
combinations  are  also  normal. .  ..Thus,  a  situation  in  which  all  variables  exhibit  univariate 
normality  will  help  gain,  although  not  guarantee,  multivariate  normality  (Hair,  Black, 
Babin,  Anderson,  &  Tatham,  2006).” 
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Using  statistics  software,  SPSS  version  16.0,  an  analysis  was  conducted  on  each 
variable  to  determine  normality.  The  results  are  summarized  in  Table  6.  A  copy  of  the 
full  analysis  for  each  variable  is  located  in  Appendix  E.  In  order  for  data  distribution  to 
be  perfectly  normal,  the  following  criteria  must  be  met  (Field,  2005): 


•  Skewness  Statistic  =  0 

•  Kurtosis  Statistic  =  0 

•  Kolmogorov-Smirnov  Test:  If  test  is  non-significant  (p  >  0.05)  then  the 
distribution  of  the  sample  data  is  not  significantly  different  from  a  normal 
distribution;  if  test  is  significant  (p  <  0.05)  then  the  distribution  of  the 
sample  is  significantly  different  from  the  normal  distribution 

•  Shapiro-Wilk  Test:  If  test  is  non-significant  (p  >  0.05)  then  the 
distribution  of  the  sample  data  is  not  significantly  different  from  a  normal 
distribution;  if  test  is  significant  (p  <  0.05)  then  the  distribution  of  the 
sample  is  significantly  different  from  the  normal  distribution 


|  Table  5.  Test  for  Normal  Distribution  j 

|  Skewness 

Kurtosis 

Kolmogorov-Smirnov  | 

Shapiro-Wilk  j 

Statistic 

Significance 

Statistic 

Significance 

Statistic 

Significance 

Statistic 

Significance 

Overall  Contractor  Performance  Rating 

0.838 

0.189 

-0.188 

0.376 

0.357 

0.000 

0.747 

0.000 

Project  Schedule  Growth 

3.034 

0.189 

13.908 

0.376 

0.287 

0.000 

0.665 

0.000 

Quality  Control 

0.038 

0.189 

0.612 

0.376 

0.161 

0.000 

0.949 

0.000 

Adequacy  of  As-Builts 

-0.612 

0.189 

-1.121 

0.376 

0.359 

0.000 

0.763 

0.000 

Effectiveness  of  Management 

0.446 

0.189 

-0.873 

0.376 

0.184 

0.000 

0.912 

0.000 

Management  of  Subcontractors 

-1.146 

0.189 

2.729 

0.376 

0.257 

0.000 

0.854 

0.000 

Timely  Performance 

-0.866 

0.189 

0.633 

0.376 

0.245 

0.000 

0.908 

0.000 

Warranty  Response 

-0.251 

0.189 

-1.322 

0.376 

0.295 

0.000 

0.812 

0.000 

Compliance  with  Labor  Standards 

-0.950 

0.189 

0.062 

0.376 

0.369 

0.000 

0.768 

0.000 

Correction  of  CLS  Noted  Deficiencies 

1.083 

0.189 

1.240 

0.376 

0.402 

0.000 

0.719 

0.000 

Compliance  with  Safety  Standards 

0.313 

0.189 

0.305 

0.376 

0.261 

0.000 

0.881 

0.000 

As  show  in  Table  6  no  variable  meets  the  criteria  for  normality.  However,  “size  has  the 
effect  of  increasing  statistical  power  by  reducing  sample  error;  the  larger  samples  sizes 
reduce  the  detrimental  effects  of  non-normality.  In  small  samples  of  50  or  fewer 
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observations,  significant  departures  from  normality  can  have  a  substantial  impact  on 
results  (Hair,  Black,  Babin,  Anderson,  &  Tatham,  2006).”  This  research  was  conducted 
on  165  data  observations;  it  is  assumed  that  while  the  data  has  a  non-normal  distribution, 
the  potential  detrimental  effects  of  error  have  been  reduced  by  the  large  sample  size. 

Assumption  2:  Homogeneity  of  Variance.  Homogeneity  of  variance  assumption 
means  “that  as  you  go  through  levels  of  one  variable,  the  variance  of  the  other  should  not 
change  (Field,  2005).”  For  this  assumption  of  normality  to  be  met,  on  a  scatter  plot  cross 
plot  of  the  regression  standardized  predicted  values  and  the  regression  standardize 
residual  should  look  like  a  random  array  of  dots  evenly  dispersed  around  zero.  Figure  3 
is  a  scatter  plot  of  the  values  used  to  determine  the  relationship  between  the  contractor’s 
overall  performance  rating  and  the  performance  elements.  While  the  random  array  of 
dots  are  dispersed  around  zero,  it  may  not  be  appropriate  to  say  they  are  evenly  dispersed 
around  zero.  Figure  4  is  a  scatter  plot  of  the  values  used  to  determine  the  relationship 
between  the  contractor’s  overall  performance  rating  and  the  overall  project  schedule 
performance.  The  random  array  of  dots  on  this  scatter  plot  are  dispersed  around  zero,  but 
again,  not  necessarily  evenly  dispersed  around  zero.  For  the  purpose  of  this  research,  it  is 
assumed  that  this  assumption  for  normality  is  generally  met. 

Assumption  3:  Interval  Data.  Interval  data  simply  means  that  the  distance 
between  points  on  a  scale  is  equal  at  all  parts  along  the  scale  (Field,  2005).  The  data  in 
this  research  is  interval  data.  Specifically,  the  overall  contractor  performance  rating  and 
all  other  ratings  associated  with  performance  sub-elements  is  given  on  a  5-point 
satisfaction  scale.  It  is  assumed  that  the  change  in  score  from  2  to  3  is  the  same  as  that 
represented  by  a  change  in  score  from  4  to  5. 


62 


Dependent  Variable:  Overall_Rating 


Regression  Standardized  Predicted  Value 

Figure  3.  Scatter  Plot  1 


Dependent  Variable:  Overall_Rating 


Assumption  4:  Independence.  Independence  simply  means  that  the  behavior  of 
one  participant  does  not  affect  the  behavior  of  another  participant  (Field,  2005).  In 
relation  to  the  data  observations  in  this  research,  the  evaluation  by  a  project  manager  for 
project  A  does  not  affect  or  influence  the  evaluation  by  a  project  manager  for  project  B. 
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As  all  four  assumptions  for  conducting  parametric  tests  were  met  for  the  data  in 
this  research,  each  of  the  research  questions  was  tested  using  multivariate  analysis. 
Research  question  1  was  tested  using  principle  component  analysis,  research  question  2 
was  tested  using  correlation  analysis  and  linear  regression,  and  research  question  3  was 
tested  using  correlation  analysis.  The  next  section  details  principle  component  analysis. 

Principle  Component  Analysis 

This  research  strived  to  have  the  most  representative  and  parsimonious  set  of 
factors  possible  to  reduce  measurement  error  and  thus  increase  the  strength  and  accuracy 
of  the  multivariate  analysis  (Hair,  Black,  Babin,  Anderson,  &  Tatham,  2006).  To  reduce 
the  number  of  independent  variables  used  in  this  research,  a  principle  component  analysis 
was  conducted  for  each  performance  element  and  corresponding  performance  items.  The 
principle  component  analysis  is  used  to  determine  if  the  observed  variables  are 
condensable  into  a  smaller  set  of  variates  (factors)  with  minimal  loss  of  information 
(Hair,  Black,  Babin,  Anderson,  &  Tatham,  2006).  For  instance,  to  determine  if 
‘correction  of  noted  deficiencies’,  ‘payrolls  properly  completed  and  submitted’,  and 
‘compliance  with  labor  laws  and  regulations’  are  all  performance  items  of  the 
performance  element  ‘compliance  with  labor  standards’,  a  principle  component  analysis 
is  used. 

A  detailed  explanation  behind  the  theory  of  principle  component  analysis  can  be 
found  in  Andy  Field’s  book,  Discovering  Statistics  Using  SPSS,  section  15.3.4.  Simply 
stated,  principle  component  analysis  starts  with  a  matrix  that  represents  the  relationship 
between  variables.  By  determining  the  eigenvalues  of  the  matrix,  the  variates,  or  linear 
components,  of  that  matrix  are  then  calculated.  Eigenvectors,  “the  elements  of  which 
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provide  the  loading  of  a  particular  variable  on  a  particular  factor  (Field,  2005),”  are 
calculated  using  the  eigenvalues. 

The  first  step  in  interpreting  results  of  a  principle  component  analysis  is 
determining  if  multicollinearity  is  a  problem  for  the  data  set.  Multicollinearity  exists 
when  two  or  more  predictor  variables  are  highly  correlated  (correlation  value  greater  than 
0.8)  and  are  indistinguishable  in  a  linear  relationship.  The  output  of  a  principle 
component  analysis  conducted  using  SPSS  16.0  includes  a  correlation  matrix.  At  the 
bottom  of  this  matrix  is  the  determinant  of  the  correlation  matrix.  If  the  determinant 
value  is  greater  than  0.0001  then  multicollinearity  is  not  a  problem  to  the  dataset  (Field, 
2005).  If,  however,  the  determinant  value  is  less  than  0.0001,  then  the  correlation  matrix 
needs  to  be  examined  for  variables  that  correlate  very  highly  (R>0.8).  If  any  items  highly 
correlate,  then  it  is  recommended  that  one  of  the  items  be  removed  from  the  principle 
component  analysis  before  proceeding.  Once  the  correlation  matrix  determinant  value 
passes  the  multicollinearity  test,  it  is  then  necessary  to  determine  how  strongly  the  factors 
group  together.  Following  the  correlation  matrix  on  an  SPSS  principle  component 
analysis  output  is  the  KMO  and  Bartlett’s  Test.  KMO  or  Kaiser-Meyer-Olkin  measure  of 
sampling  adequacy  is  used  to  identify  if  a  set  of  variables,  when  factored  together,  yield 
distinct  and  reliable  factors  (Field,  2005).  KMO  statistics  vary  between  values  of  0  to  1. 
Kaiser  suggests  that  accepting  values  greater  than  0.5  is  barely  acceptable;  values 
between  0.5  and  0.7  are  mediocre;  values  between  0.7  and  0.8  are  good;  values  between 
0.8  and  0.9  are  great;  and  values  greater  than  0.9  are  superb  (Field,  2005).  The  next 
output  from  SPSS  for  a  principle  component  analysis  is  a  component  matrix.  This 
component  matrix  is  used  to  determine  which  factors  group  into  each  specific 
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component.  Every  factor  included  in  a  principle  component  analysis  will  carry  a  loading 


on  every  resulting  component.  Each  factor  aligns  best  with  the  component  for  which  it 
has  the  highest  loading  (Field,  2005).  For  instance.  Table  6  is  an  example  of  a 
component  matrix  resulting  from  a  principle  component  analysis.  Interpretation  of  this 
table  indicates  that  there  are  two  components  that  resulted  from  the  principle  component 
analysis  of  nine  factors.  Based  on  the  rules  for  grouping  factors,  all  the  factors,  with  the 
exception  of  QC.i  are  grouped  into  component  1;  factor  QC.i  is  grouped  into  component 


2. 


Table  6.  Example  Principle  Component  Analysis  -  Component  Matrix 

Factor 

Component 

1 

2 

QC.a 

0.639 

-0.303 

QC.b 

0.732 

-0.385 

QC.c 

0.751 

-0.430 

QC.d 

0.723 

-0.344 

QC.e 

0.619 

0.365 

QC.f 

0.743 

0.044 

QC.g 

0.754 

0.173 

QC.h 

0.666 

0.366 

QC.i 

0.331 

0.650 

QC.j 

0.688 

0.179 

QC.k 

0.584 

0.131 

Research  Questions 

The  objective  of  this  research  was  to  determine  if  the  performance  elements  and 
performance  items  listed  on  DD  Form  2626  represent  the  overall  performance  of  the 
construction  contractor.  Therefore, 
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Research  Question  1:  Do  the  DD  Form  2626  performance  elements  reflect  their 
respective  performance  items? 

Figure  5  is  a  conceptual  representation  of  research  question  1. 


Performance  Items 

Performance  Elements 

QC.a 

Quality  of  service 

QC.b 

Adequacy  of  CQC  plan 

QC.c 

Implementation  of  CQC  plan 

QC.d 

Quality  of  QC  documentation 

QC.e 

Storage  of  materials 

QC.f 

Adequacy  of  materials 

RQ#1 

QC.g 

Adequacy  of  submittals 

— ^  QC  -  Quality  Control 

QC.h 

Adequacy  of  QC  testing 

QC.i 

Adequacy  of  as-builts 

QC.j 

Use  of  specified  material 

QC.k 

Identification/correction  of 

deficient  work  in  a  timely 

manner 

EM.a 

Cooperation  and  responsiveness 

EM.b 

Management  of  resources/ 

personnel. 

EM.c 

Coordination  and  control  of 

subcontractors) 

EM.d 

Adequacy  of  site  clean  up 

EM.e 

Effectiveness  of  job-site 

RQ#1 

supervision 

— ^  EM  -  Effectiveness  of  Management 

EM.f 

Compliance  with  laws  and 

regulations 

EM.g 

Professional  conduct 

EM.h 

Review/resolution  of 

subcontractor  issues 

EM.i 

Implementation  of  subcontractor 

plan 

TP. a 

Adequacy  of  initial  progress  schedule 

TP.b 

Adherence  to  approved  schedule 

TP.c 

Resolution  of  delays 

TP.d 

Submission  of  required  documentation 

RQ#1 

TP.e 

Completion  of  punchlist  items 

— ^  TP  -  Timely  Performance 

TP.f 

Submission  of  updated  and  revised 

progress  schedules 

TP  .g 

Warranty  response 

CLS.a 

Correction  of  noted  deficiencies 

CLS.b 

Payrolls  properly  completed  and 

submitted 

RQ#1 

CLS.c 

Compliance  with  labor  laws  and 

— ^  CLS  -  Compliance  with  Labor  Standards 

regulations  with  specific  attention  to 

the  Davis-Bacon  Act  and  EEO 

requirements 

CSS.a 

Adequacy  of  safety  plan 

RQ#1 

CSS.b 

Implementation  of  safety  plan 

— ^  CSS  -  Compliance  with  Safety  Standards 

CSS.c 

Correction  of  noted  deficiencies 

Figure  5.  Conceptual  Model  of  Research  Question  1 
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Principle  component  analysis  was  conducted  on  the  five  performance  elements  and 
respective  performance  items  using  SPSS  16.0.  Limited  information  is  available  in  the 
literature  to  support  the  current  categorization  of  sub-performance  elements  as  they 
appear  on  the  DD  Form  2626.  Face  validity  will  serve  to  support  maintaining  each 
performance  sub-element  with  its  respective  performance  element  when  executing  the 
initial  principle  component  analysis.  “By  default,  SPSS  uses  Kaiser’s  criterion  for 
extracting  factors  (Field,  2005).”  Kaiser’s  criterion  recommends  that  all  factors  with 
eigenvalues  greater  than  1  be  retained.  Results  of  the  principle  component  analysis  are 
discussed  in  Chapter  4  of  this  research. 

Research  Question  2:  Do  the  performance  items  as  they  are  appropriately 
aggregated  into  performance  elements  predict  the  contractor ’s  overall 
performance  rating? 

Figure  6  is  a  conceptual  representation  of  research  question  2.  Using  the  results  from  the 
component  analysis,  the  performance  elements  and  the  overall  contractor  performance 
rating  were  individually  correlated  to  determine  if  a  relationship  exists.  Once 
relationships  were  established,  logistical  regression  analysis  was  conducted  using  SPSS 
16.0  to  determine  the  combination  of  specific  performance  elements  that  result  in 
predicting  the  contractor’s  overall  performance  rating.  SPSS  16.0  also  facilitated 
specification  of  a  stepwise  procedure  for  the  selection  of  independent  variables  (from  the 
performance  elements  developed  in  the  principle  component  analysis)  and  goodness-of- 
fit  statistics  for  the  developed  contractor  performance  prediction  model. 
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Performance 

Elements 


Overall 

Performance 


RQ  #2 

- ► 

Figure  6.  Conceptual  Model  of  Research  Question  2 

As  mentioned  earlier,  a  contractor’s  overall  project  performance  is  based  on 
quality,  schedule,  and  cost  (Ling  &  Liu,  2004).  To  test  if  this  statement  is  true,  the 
Overall  Contractor  Rating  was  compared  to  the  Project  Schedule  Growth  to  determine  if 
a  correlation  exists  between  a  contractor’s  perfonnance  and  schedule  compliance.  If 
USACE  project  managers  truly  value  schedule  perfonnance,  then  the  contractor’s  overall 
performance  rating  should  decrease  as  the  project  schedule  growth  increases.  Therefore, 
Research  Question  3:  Is  the  contractor ’s  overall  performance  rating  and  the 
overall  project  schedule  performance  related? 

Figure  7  is  a  conceptual  representation  of  research  questions  3. 


Figure  7.  Conceptual  Model  of  Research  Questions  3 

Study  Significance  Level 

Significance  level  (alpha)  refers  to  the  risk  the  researcher  is  willing  to  take  that 
the  estimated  coefficient  is  not  statistically  different  from  zero  (Hair,  Black,  Babin, 
Anderson,  &  Tatham,  2006).  This  is  referred  to  as  a  Type  I  error.  Typically,  a  value  of 
.05  is  used  for  comparing  the  level  of  significance  with  the  p-value  (or  observed 
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significance).  For  this  study,  a  one-tailed  test  statistic  that  produces  a  p-value  less  than 
0.05  will  be  considered  statistically  significant  and  highly  significant  if  less  than  0.01 
(Hair,  Black,  Babin,  Anderson,  &  Tatham,  2006). 

Summary 

Chapter  3,  Methodology,  described  the  data  source,  data  collection,  and  data 
analysis  methods  that  were  used  to  analyze  the  methods  employed  by  the  USACE  to 
evaluation  and  rate  the  performance  of  construction  contractors.  A  principle  component 
analysis  was  conducted  on  each  performance  item  as  they  are  grouped  on  DD  Form  2626 
to  determine  if  the  performance  items  represent  their  respective  performance  elements. 
The  result  of  the  principle  component  analysis  identified  more  representative 
performance  elements  which  were  then  used  to  identify  if  a  relationship  exists  between 
the  performance  elements  and  the  contractor’s  overall  performance  rating.  Finally,  the 
contractor’s  overall  performance  rating  was  compared  with  the  overall  schedule 
performance  to  determine  if  traditional  measures  of  project  success  are  actively  used  in 
the  USACE’s  evaluation  process.  Chapter  4,  Results,  will  present,  analyze,  and  discuss 
the  data  in  order  to  answer  the  research  questions. 
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IV.  Analysis  and  Results 


Chapter  Overview 

This  chapter  provides  the  raw  results  of  the  study  based  on  the  methodology 
developed  in  Chapter  3.  The  results  are  presented  graphically  and  in  tables  with 
discussions  limited  to  the  statistical  analysis.  Presentation  of  the  research  results  follows 
the  research  questions  developed  in  Chapter  1  and  reiterated  in  Chapter  3.  Additional 
interpretation,  explanation,  and  speculation  of  the  results  will  be  addressed  in  Chapter  5: 
Conclusions. 

Results 

The  objective  of  this  research  is  to  strengthen  the  USAF  rating  system  of 
contractor  performance  by  evaluating  DD  Form  2626  Construction  Contractor 
Performance  Evaluation.  The  United  States  Army  Corp  of  Engineers  uses  the  DD  Form 
2626  to  evaluate  and  rate  construction  contractors’  performance  and  then  uses  the 
evaluations  for  future  source  selections.  The  research  analyzed  DD  Form  2626  to 
determine  if  any  of  the  performance  items  are  correlated  to  a  contractor’s  overall 
performance  rating.  The  research  also  determined  that  a  relationship  exists  between 
overall  schedule  performance  and  a  contractor’s  overall  performance.  Figure  8 
summarizes  the  research  into  the  relationships  of  this  study. 
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Performance  Items 

Performance  Elements 

QC-a 

Quality  of  service 

QC.b 

Adequacy  of  CQC  plan 

QC.c 

Implementation  of  CQC  plan 

QC.d 

Quality  of  QC  documentation 

QC.e 

Storage  of  materials 

QC.f 

Adequacy  of  materials 

RQ#1 

QC.g 

Adequacy  of  submittals 

QC  -  Quality  Control 

QC.h 

Adequacy  of  QC  testing 

QC.i 

Adequacy  of  as-builts 

QC.j 

Use  of  specified  material 

QC.k 

Identification/correction  of 

deficient  work  in  a  timely 

manner 

EM.a 

Cooperation  and  responsiveness 

EM.b 

Management  of  resources/ 

personnel. 

EM.c 

Coordination  and  control  of 

subcontractor!  s) 

EM.d 

Adequacy  of  site  clean  up 

EM.e 

Effectiveness  of  job-site 

RQ#1 

supervision 

EM  -  Effectiveness  of  Management 

EM.f 

Compliance  with  laws  and 

regulations 

EM.g 

Professional  conduct 

EM.h 

Review/resolution  of 

subcontractor  issues 

EM.i 

Implementation  of  subcontractor 

plan 

TP.a 

Adequacy  of  initial  progress  schedule 

TP.b 

Adherence  to  approved  schedule 

TP.c 

Resolution  of  delays 

TP.d 

Submission  of  required  documentation 

TP.e 

Completion  of  punchlist  items 

-►  TP  -  Timely  Performance 

TP.f 

Submission  of  updated  and  revised 

progress  schedules 

TP.g 

Warranty  response 

CLS.a 

Correction  of  noted  deficiencies 

CLS.b 

Payrolls  properly  completed  and 

submitted 

RQ#1 

Compliance  with  labor  laws  and 

CLS  -  Compliance  with  Labor  Standards 

regulations  with  specific  attention  to 

the  Da vis-Bacon  Act  and  EEO 

requirements 

Overall  |  RQ  #3 
Performance 


Overall 

Project  Schedule 
Performance 


Adequacy  of  safety  plan 
Implementation  of  safety  plan 
Correction  of  noted  deficiencies 


'  CSS  -  Compliance  with  Safety  Standards 


Figure  8.  Relationship  Conceptual  Model 

Research  Question  1:  Do  the  DD  Form  2626 performance  elements  reflect  their 
respective  performance  items? 

Reliability  of  the  measured  data  was  first  obtained  using  SPSS  16.0.  The  most 
common  measure  of  scale  reliability  is  Cronbach’s  alpha.  Generally  a  value  of  0.7  -  0.8 
is  an  acceptable  value  for  Cronbach’s  alpha  (Field,  2005).  The  Cronbach’s  alpha  values 
for  each  of  the  performance  elements  are  as  follows:  Quality  Control  =  0.846, 
Effectiveness  of  Management  =  0.839,  Timely  Performance  =  0.810,  Compliance  with 
Labor  Standards  =  0.268,  and  Compliance  with  Safety  Standards  =  0.729.  All  the 
performance  elements  have  acceptable  Cronbach’s  alpha  values  except  Compliance  with 
Labor  Standards.  Reasons  for  this  will  be  discussed  in  detail  in  Chapter  5.  Results  of  the 
reliability  analysis  can  be  found  in  Appendix  L. 
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A  principle  component  analysis  was  conducted  to  test  if  the  five  performance 


elements  provided  on  DD  Form  2626  are  represented  by  their  respective  performance 
items.  Details  on  the  method  of  principle  component  analysis  can  be  found  in  Chapter  3. 
Recall:  When  conducting  a  principle  component  analysis,  before  factors  are  aggregated 
by  component,  it  should  first  be  determined  if  the  data  set  is  at  risk  of  multicollinearity. 

If  the  determinant  value  is  greater  than  0.0001,  then  the  dataset  is  not  at  risk  of 
multicollinearity  (Field,  2005).  The  KMO  value  will  help  determine  if  the  set  of 
variables,  when  factored  together,  yield  distinct  and  reliable  factors.  KMO  statistics  vary 
between  values  of  0  to  1;  Kaiser  suggests  that  accepting  values  greater  than  0.5  are  barely 
acceptable;  values  between  0.5  and  0.7  are  mediocre;  values  between  0.7  and  0.8  are 
good;  values  between  0.8  and  0.9  are  great;  and  values  greater  than  0.9  are  superb  (Field, 
2005). 

A  principle  component  analysis  and  confirmatory  component  analysis  were 
conducted  on  all  33  items  before  each  of  the  five  performance  element  groupings  were 
tested  to  see  how  each  of  the  items  would  factor  together.  This  analysis  produced  seven 
components.  Table  7  summarizes  the  results  of  the  principle  component  analysis.  None 
of  the  performance  items  factored  together  with  their  respective  performance  element,  as 
they  are  grouped  on  DD  Form  2626.  Additionally,  the  dataset  as  a  whole  suffers  from 
multicollinearity  with  a  determinate  value  of  1.17E-11  -  a  value  that  is  seven  orders  of 
magnitude  less  than  the  threshold  criteria.  However,  the  KMO  value  is  0.856  indicating 
that  factoring  the  variables  together  should  yield  “great”  distinct  and  reliable  factors.  For 
the  confirmatory  factor  analysis,  five  components  were  indicated  for  the  factor  analysis. 


73 


| Table  7:  Overall  Principle  Component  Analysis  | 

Component 

Component 

Performance  Item 

Number 

Performance  Item 

Number 

QC.a  -  Quality  of  Workmanship 

EM.i  -  Implementation  of  Subcontracting  Plan 

QC.b  -  Adequacy  of  the  CQC  Plan 

TP.f  -  Submission  of  Updated  and  Revised  Progress 

2 

QC.c  -  Implementation  of  the  CQC  Plan 

Schedules 

QC.d  -  Quality  of  QC  Documentation 

QC.f  -  Adequacy  of  Materials 

CSS.c  -  Correction  of  Noted  Deficiencies 

QC.g  -  Adequacy  of  Submittals 

CLS.a  -  Correction  of  Noted  Deficiencies 

QC.h  -  Adequacy  of  Testing 

QC.k  -  Identification/Correction  of  Deficient  Work  in 

QC.e  -  Storage  of  Materials 

4 

EM. a  -  Cooperation  and  Responsiveness 

EM.b  -  Management  of  Resources  Personnel 

CLS.c  -  Compliance  with  Labor  Laws  and 

EM.c  -  Coordination  and  Control  of  Subcontract(s) 

Regulations  with  Specific  Attention  to  the  Davis- 

5 

EM.d  -  Adequacy  of  Site  Clean  Up 

i 

Bacon  Act  and  EEO  Regulations 

EM.e  -  Effectiveness  of  Job-Site  Supervision 

EM.f  -  Compliance  with  Laws  and  Regulations 

QC.i  -  Adequacy  of  As-Builts 

6 

EM.g  -  Professional  Conduct 

EM.h  -  Review/Resolution  of  Subcontractor's  Issues 

TP.g  -  Warranty  Response 

7 

TP.  a  -  Adequacy  of  Initial  Progress  Schedule 

TP.b  -  Adherence  to  Approved  Schedule 

TP.c  -  Resolution  of  Delays 

TP.d  -  Submission  of  Required  Documentation 

TP.e  -  Completion  of  Punchlist  Items 

CLS.b  -  Payrolls  Properly  Completed  and  Submitted 
CSS. a  -  Adequacy  of  Safety  Plan 

CSS.b  -  Implementation  of  Safety  Plan 

While  confirmatory  factor  analysis  resulted  in  five  components,  none  of  the  performance 
items  factored  together  with  their  respective  performance  element,  as  they  are  grouped  on 
DD  Form  2626.  Therefore,  for  the  remaining  principle  component  analyses,  the 
performance  items  will  be  analyzed  as  they  are  grouped  on  DD  Form  2626. 

The  first  performance  element  analyzed  was  Quality  Control.  Figure  9  represents 
the  analysis  of  the  relationship  between  Quality  Control  and  its  respective  performance 
items.  Quality  Control  is  represented  by  eleven  performance  items  on  DD  Form  2626. 
The  principle  component  analysis  resulted  in  two  components.  The  following 
performance  items  grouped  into  component  1  and  therefore  were  found  to  be 
representative  of  performance  element  Quality  Control  1:  Quality  of  Workmanship, 
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Adequacy  of  the  CQC  Plan,  Implementation  of  the  CQC  Plan,  Quality  of  QC 
Documentation,  Adequacy  of  Materials,  Adequacy  of  Submittals,  Adequacy  of  Testing, 
and  Identification/Correction  of  Deficient  Work  in  a  timely  manner.  Component  2 
captured  three  performance  items  -  Storage  of  Materials,  Adequacy  of  As-Builts,  and 
Use  of  Specified  Materials.  For  analysis  of  research  question  number  two,  eight  of  the 
performance  items  listed  under  Quality  Control  were  averaged  together  to  represent 
performance  element  Quality  Control  1.  Performance  items  Storage  of  Materials, 
Adequacy  of  As-Builts,  and  Use  of  Specified  Materials  were  averaged  together  to 
represent  performance  element  Quality  Control  2.  The  initial  and  confirmatory  results  of 


the  principle  component  analysis  for  Quality  Control  are  located  in  Appendix  H. 


Performance  Items 

Performance  Elements 

QC.a 

Quality  of  service 

QC.b 

Adequacy  of  CQC  plan 

QC.c 

Implementation  of  CQC  plan 

QC.d 

Quality  of  QC  documentation 

QC.e 

Storage  of  materials 

QC.f 

Adequacy  of  materials 

- - ►  QC  -  Quality  Control 

QC.g 

Adequacy  of  submittals 

QC.h 

Adequacy  of  QC  testing 

QC.i 

Adequacy  of  as-builts 

QC.j 

Use  of  specified  material 

QC.k 

Identification/correction  of 
deficient  work  in  a  timely 
manner 

Figure  9.  Conceptual  Model  of  Research  Question  1  -  Quality  Control 


The  second  performance  element  analyzed  was  Effectiveness  of  Management. 
Figure  10  represents  the  analysis  of  the  relationship  between  Effectiveness  of 
Management  and  its  respective  performance  items.  On  DD  Form  2626,  Effectiveness  of 
Management  is  represented  by  nine  performance  items.  The  principle  component 
analysis  resulted  in  two  components.  The  following  performance  items  grouped  into 
component  1  and  therefore  were  found  to  be  representative  of  performance  element 
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Effectiveness  of  Management:  Cooperation  and  Responsiveness,  Management  of 
Resources/Personnel,  Adequacy  of  Site  Clean-Up,  Effectiveness  of  Job-Site  Supervision, 
Compliance  with  Laws  and  Regulations,  and  Professional  Conduct.  The  remaining  three 
performance  items  listed  under  Effectiveness  of  Management  grouped  into  component  2. 
They  are  Coordination  and  Control  of  Subcontractor(s),  Review/Resolution  of 
Subcontractor’s  Issues,  and  Implementation  of  Subcontracting  Plan.  It  is  not  surprising 
that  subcontractor  performance  items  do  not  factor  with  the  other  effectiveness  of 
management  performance  items.  Of  the  165  projects  analyzed  in  this  research,  only  65% 
of  the  projects  included  subcontractors.  It  is  appropriate  that  the  subcontractor 
performance  items  factor  together  and  are  exclusive  of  the  other  Effectiveness  of 
Management  performance  items.  To  address  research  question  number  two  the 
performance  items  listed  under  Effectiveness  of  Management,  excluding  the  three 
performance  items  mentioned,  were  averaged  together  to  represent  performance  element 
Effectiveness  of  Management.  Performance  items  Coordination  and  Control  of 
Subcontractor(s),  Review/Resolution  of  Subcontractor’s  Issues,  and  Implementation  of 
Subcontracting  Plan  were  averaged  together  to  create  a  new  performance  element  titled 
Management  of  Subcontractor(s).  The  initial  and  confirmatory  results  of  the  principle 
component  analysis  for  effectiveness  of  management  are  located  in  Appendix  I. 
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Performance  Items 

Performance  Elements 

EM.a 

Cooperation  and  responsiveness 

EM.b 

Management  of  resources/ 
personnel, 

EM.c 

Coordination  and  control  of 
subcontractors) 

EM.d 

Adequacy  of  site  clean  up 

EM.e 

Effectiveness  of  job-site 

RQ#1 

- — - p  EM  -  Effectiveness  of  Management 

supervision 

EM.f 

Compliance  with  laws  and 
regulations 

EM.g 

Professional  conduct 

EM.h 

Review/resolution  of 
subcontractor  issues 

EM.i 

Implementation  of  subcontractor 
plan 

Figure  1 0.  Conceptual  Model  of  Research  Question  1 :  Effectiveness  of  Management 


The  third  performance  element  analyzed  was  Timely  Performance.  Figure  1 1 
represents  the  analysis  of  the  relationship  between  Timely  Performance  and  its  respective 
performance  items.  On  DD  Form  2626,  Timely  Performance  is  represented  by  seven 
performance  items.  The  principle  component  analysis  resulted  in  two  components.  The 
following  performance  items  grouped  into  component  1  and  were  found  to  be 
representative  of  performance  element  Timely  Performance:  Adequacy  of  Initial 
Progress  Schedule,  Adherence  to  Approved  Schedule,  Resolution  of  Delays,  Submission 
of  Required  Documentation,  Completion  of  Punchlist  Items,  and  Submission  of  Updated 
and  Revised  Progress  Schedules.  Component  2  only  captures  one  performance  item  - 
Warranty  Response.  Not  all  construction  projects  were  rated  for  Warranty  Response.  Of 
the  165  projects  in  the  data  set  used  in  the  analysis,  only  67%  were  applicable  for 
Warranty  Response  evaluation.  This  may  account  for  Warranty  Response  factoring 
separately  from  the  other  six  Timely  Response  performance  items.  To  address  research 
question  two  the  performance  items  listed  under  Timely  Response,  excluding  Warranty 
Response,  were  averaged  together  to  represent  performance  element  Timely  Response. 
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Performance  item  Warranty  Response  was  singled  out  as  its  own  performance  element. 
The  initial  and  confirmatory  results  of  the  principle  component  analysis  for  timely 
performance  are  located  in  Appendix  J. 


Performance  Items 

Performance  Elements 

TP. a 

Adequacy  of  initial  progress  schedule 

TP.b 

Adherence  to  approved  schedule 

TP.c 

Resolution  of  delays 

TP.d 

Submission  of  required  documentation 

RQ#1 

TP.e 

Completion  of  punchlist  items 

- - - ►  TP  -  Timely  Performance 

TP.f 

Submission  of  updated  and  revised 

progress  schedules 

TP  .g 

Warranty  response 

Figure  1 1 .  Conceptual  Model  of  Research  Question  1  -  Timely  Performance 


The  fourth  performance  element  analyzed  was  Compliance  with  Labor  Standards. 
Figure  12  represents  the  analysis  of  the  relationship  between  Compliance  with  Labor 
Standards  and  its  respective  performance  items.  On  DD  Form  2626,  Compliance  with 
Labor  Standards  is  represented  by  three  performance  items.  The  following  performance 
items  were  found  representative  of  the  performance  element  Compliance  with  Labor 
Standards  and  were  grouped  into  component  1:  Payrolls  Properly  Completed  and 
Submitted,  and  Compliance  with  Labor  Laws  and  Regulations  with  Specific  Attention  to 
the  Davis-Bacon  Act  and  EEO  Requirements.  Component  2  only  captures  one 
performance  item  -  Correction  of  Noted  Deficiencies.  For  analysis  of  research  question 
number  two,  all  the  performance  items  except  Correction  of  Noted  Deficiencies  were 
averaged  together  to  represent  performance  element  Compliance  with  Labor  Standards. 
The  performance  item  Correction  of  Noted  Deficiencies  was  singled  out  as  its  own 
performance  element  titled  Correction  of  Compliance  with  Labor  Standards  (CLS)  Noted 
Deficiencies.  The  initial  and  confirmatory  results  of  the  principle  component  analysis  for 
compliance  with  labor  standards  are  located  in  Appendix  K. 
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Performance  Items 

Performance  Elements 

CLS.a 

Correction  of  noted  deficiencies 

CLS.b 

Payrolls  properly  completed  and 

CLS.c 

submitted 

Compliance  with  labor  laws  and 
regulations  with  specific  attention  to 
the  Da vis-Bacon  Act  and  EEO 

RQ#1 

- - - ^  CLS  -  Compliance  with  Labor  Standards 

requirements 

Figure  12.  Conceptual  Model  of  Research  Question  1  -  Compliance  with  Labor  Standards 


The  fifth  performance  element  analyzed  was  Compliance  with  Safety  Standards. 
Figure  13  represents  the  analysis  of  the  relationship  between  Compliance  with  Safety 
Standards  and  its  respective  performance  items.  On  DD  Form  2626,  Compliance  with 
Safety  Standards  is  represented  by  three  performance  items.  All  the  performance  items 
listed  under  Compliance  with  Safety  Standards  were  found  to  be  representative  and 
grouped  into  one  component.  These  performance  items  are  Adequacy  of  Safety  Plan, 
Implementation  of  Safety  Plan,  and  Correction  of  Noted  Deficiencies.  All  the 
performance  items  listed  under  compliance  with  safety  standards  were  averaged  together 
to  represent  performance  element  Compliance  with  Safety  Standards.  This  will  be  use 
din  the  analysis  of  research  question  two.  The  initial  and  confirmatory  results  of  the 
principle  component  analysis  for  timely  performance  are  located  in  Appendix  L. 


Performance  Items 

Performance  Elements 

CSS. a 

Adequacy  of  safety  plan 

RQ#i 

CSS.b 

Implementation  of  safety  plan 

- - - ►  CSS  -  Compliance  with  Safety  Standards 

CSS.c 

Correction  of  noted  deficiencies 

Figure  13.  Conceptual  Model  of  Research  Question  1  -  Compliance  with  Safety  Standards 


The  overall  result  of  the  principle  component  analysis  was  a  reduction  of  33 
independent  variables  to  nine  independent  variables:  Quality  Control  1  (QCl),  Quality 
Control  2  (QC2),  Effectiveness  of  Management  (EM),  Management  of  Subcontractors 


79 


(MS),  Timely  Performance  (TP),  Warranty  Response  (WR),  Compliance  with  Labor 
Standards  (CLS),  Correction  of  CLS  Noted  Deficiencies  (CCLSND),  and  Compliance 
with  Safety  Standards  (CSS).  Table  8  summarizes  the  determinant  and  KMO  values  for 
each  of  the  nine  performance  elements.  Notice  that  all  the  factored  performance  elements 
exceed  the  determinant  value  for  multicollinearity  (R  >  0.0001)  and  are  considered  to 
yield  an  “acceptable”  distinct  and  reliable  factor. 


Table  8.  Summary  of  Determinant  and  KMO  Values  for  Finalized  Performance  Elements 

QC1 

QC2 

EM 

TP 

WR 

CLS 

CCLSND 

CSS 

Determinant 

0.023 

0.688 

0.017 

NA* 

0.336 

NA 

0.434 

KMO  Value 

0.785 

0.555 

0.898 

0.632 

0.842 

NA* 

0.500 

NA 

0.650 

*  No  determinant  or  KMO  values  because  these  represent  single  peformance  items  turned  into  individual  performance  elements 


These  nine  acceptable  factored  performance  elements  were  used  as  independent  variables 
to  analyze  research  question  number  two  discussed  in  detail  in  the  next  section. 


Research  Question  2:  Do  the  performance  items  as  they  are  appropriately  aggregated 
together  into  performance  elements  predict  the  contractor’s  overall  performance 
rating? 

Using  SPSS  version  16.0,  a  correlation  analysis  was  conducted  to  determine  if  a 
relationship  exists  between  the  nine  factored  performance  elements  derived  from  the 
principle  component  analysis  and  the  contractor’s  overall  performance  rating.  Figure  14 
is  a  conceptual  model  of  the  proposed  relationship  between  the  factored  performance 
elements  and  the  contractor’s  overall  performance  rating.  A  summary  of  the  correlation 
results  is  provided  in  Table  9.  Detailed  results  of  each  correlation  analysis  are  provided 
in  Appendix  M.  Eight  of  the  nine  performance  elements  analyzed  have  a  significant 
relationship  with  the  contractor’s  overall  performance  rating.  Effectiveness  of 
Management  has  the  strongest  correlation  to  the  contractor’s  overall  performance  rating, 
whereas,  Warranty  Response  has  no  relationship  to  the  contractor’s  overall  performance 
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Performance  Elements 


QC1  -  Quality  Control  1 

QC2  -  Quality  Control  2 

EM- Effectiveness  of  Management 

MS  -  Management  of  Subcontractors 

TP  -  Timely  Performance 

WR  -  Warranty  Response 

CLS  -  Compliance  with  Labor  Standards 

CCLSND  -  Correction  of  CLS  Noted  Deficiencies 

CSS  -  Compliance  with  Safety  Standards 


RQ  #2 


> 


Overall 

Performance 


Figure  1 4.  Conceptual  Model  of  Research  Question  2 

rating.  At  first  glance  it  would  appear  that  Effectiveness  of  Management,  as  aggregated 
from  the  appropriate  performance  items,  has  the  strongest  influence  on  the  contractor’s 
overall  performance  rating.  It  also  appears  that  the  contractor’s  warrantee  response  and 
the  contractor’s  correction  of  noted  deficiencies  in  compliance  with  labor  standards  have 
little  to  no  influence  on  the  contractor’s  overall  performance  rating.  Correlation  analysis 
only  considers  the  relationship  between  each  individual  performance  element  and  the 
contractor’s  overall  performance  rating.  The  next  step  in  this  analysis  was  to  conducted  a 
step-wise  linear  regression  in  order  to  understand  how  the  performance  elements  interact 
with  each  other  to  affect  the  contractor’s  overall  performance  rating. 


|  Table  9.  Correlation  Analysis  of  Performance  Elements  and  Contractor's  Overall  Performance  Rating  | 

QCl 

QC2 

EM 

MS 

TP 

WR 

CLS 

CCLSND 

CSS 

Contractor's  Overall 
Performance  Rating 

0.628** 

0.219** 

0.869** 

0.380** 

0.448** 

0.098 

0.528** 

0.156* 

0.535** 

*  Correlation  is  significant  at  the  0.05  level 
**  Correlation  is  significant  at  the  0.01  level 


Using  SPSS  version  16.0,  a  step-wise  linear  regression  was  conducted  on  those 
performance  elements  with  a  significant  relationship  to  the  contractor’s  overall 
performance  rating  to  determine  if  in  fact  the  contractor’s  overall  performance  rating  is  a 
composite  score  of  the  measured  performance  items  as  aggregated  into  performance 
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elements.  The  step-wise  linear  regression  resulted  in  two  models.  Detailed  can  be  found 
in  Appendix  N. 

Figure  15  is  the  first  mathematical  model  that  resulted  from  the  linear  regression. 
It  has  one  predictor,  EM  =  effectiveness  of  management,  and  a  R“  value  of  0.755.  This 
means  that  75.5%  of  the  variance  in  the  contractor’s  overall  performance  rating  is 
accounted  for  by  effectiveness  of  management  averaged  ratings.  Using  the  50  project 
evaluations  that  were  randomly  removed  from  the  original  215  project  evaluations, 
mathematical  Model  #1  was  tested  for  its  accuracy  its  ability  to  predict  the  actual 
contractor’s  overall  performance  rating.  Table  10  summarizes  the  prediction  results  for 
the  50  project  evaluations  set  aside  for  model  validation.  Mathematical  Model  #1 
accurately  predicted  44  of  50  contractor’s  overall  performance  ratings.  This  gives  the 
model  a  reliability  of  88%.  The  model  will  accurately  predict  the  contractor’s  overall 
performance  rating  only  using  the  Effectiveness  of  Management  score  88%  of  the  time. 
This  score  is  an  average  of  the  ratings  given  for  the  performance  items:  Cooperation  and 
responsiveness,  management  of  resources/personnel,  adequacy  of  site  clean-up, 
effectiveness  of  job-site  supervision,  compliance  with  laws  and  regulations,  and 
professional  conduct. 

Y=  0.176+  0.936X; 

where 

Y  =  Contractor  s  Overall  Performance  Rating  (Predicted) 

Xj  =  Effectiveness  of  Management  (EM) 

Figure  15:  Mathematical  Model  #1 
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Table  10:  Model  Validation  EM 


Project 

Overall  Rating 
Actual 

Overall  Rating 
Predicted 

Project 

Overall  Rating 
Actual 

Overall  Rating 
Predicted 

Project  1 

3 

3 

Project  26 

3 

3 

Project  2 

3 

3 

Project  27 

3 

3 

Project  3 

3 

3 

Project  28 

4 

4 

Project  4 

4 

4 

Project  29 

3 

3 

Project  5 

3 

4 

Project  30 

3 

3 

Project  6 

3 

3 

Project  3 1 

4 

4 

Project  7 

4 

4 

Project  32 

4 

3 

Project  8 

4 

4 

Project  33 

3 

3 

Project  9 

3 

3 

Project  34 

2 

3 

Project  10 

4 

4 

Project  35 

3 

3 

Project  11 

3 

3 

Project  36 

3 

3 

Project  12 

4 

4 

Project  37 

3 

3 

Project  13 

3 

3 

Project  38 

5 

5 

Project  14 

4 

4 

Project  39 

3 

3 

Project  15 

4 

4 

Project  40 

4 

4 

Project  16 

4 

3 

Project  41 

4 

4 

Project  17 

4 

4 

Project  42 

3 

3 

Project  18 

3 

3 

Project  43 

3 

3 

Project  19 

3 

3 

Project  44 

3 

4 

Project  20 

3 

3 

Project  45 

3 

3 

Project  2 1 

3 

4 

Project  46 

3 

3 

Project  22 

4 

4 

Project  47 

3 

3 

Project  23 

3 

3 

Project  48 

3 

3 

Project  24 

4 

4 

Project  49 

3 

3 

Project  25 

4 

4 

Project  50 

3 

3 

Figure  16  is  the  second  mathematical  model  that  resulted  from  the  linear 
regression.  It  has  two  predictors,  EM  =  effectiveness  of  management  and  TP  =  timely 
performance,  and  a  R"  value  of  0.766.  This  means  that  76.6%  of  the  variance  in  the 
contractor’s  overall  performance  rating  is  accounted  for  by  the  combination  of 
effectiveness  of  management  averaged  ratings  and  timely  performance  averaged  ratings. 
Mathematical  Model  #2  was  tested  for  its  accuracy  in  predicting  the  actual  contractor’s 
overall  performance  rating  using  the  50  randomly  sequestered  project  evaluations  from 
the  original  215  project  evaluations.  Table  1 1  summarizes  the  prediction  results  for  the 
50  project  evaluations  set  aside  for  model  validation.  Mathematical  Model  #2  accurately 
predicted  45  of  50  contractor’s  overall  performance  ratings,  giving  the  model  a  reliability 
of  90%.  This  means  that  90%  of  the  time,  the  model  will  accurately  predict  the 
contractor’s  overall  performance  rating  using  the  Effectiveness  of  Management  (EM) 
score  and  the  Timely  Performance  (TP)  score.  Derivation  of  the  EM  score  was  discussed 
previously;  TP  is  derived  by  averaging  of  the  ratings  for  the  following  performance 
items:  Adequacy  of  initial  progress  schedule,  adherence  to  approved  schedule,  resolution 
of  delays,  submission  of  required  documentation,  completion  of  punchlist  items,  and 
submission  of  updated  and  revised  progress  schedules. 

Y=  0.116  +  0. 885 +  0.083A2 

where 

Y  =  Contractor  s  Overall  Performance  Rating  (Predicted) 

X]  =  Effectiveness  of  Management  (EM) 

X2  =  Timely  Performance  (TM) 

Figure  16:  Mathematical  Model  #2 
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Table  11:  Model  Validation  EM  &  TP 


Project 

Overall  Rating 
Actual 

Overall  Rating 
Predicted 

Project 

Overall  Rating 
Actual 

Overall  Rating 
Predicted 

Project  1 

3 

3 

Project  26 

3 

3 

Project  2 

3 

3 

Project  27 

3 

3 

Project  3 

3 

3 

Project  28 

4 

4 

Project  4 

4 

4 

Project  29 

3 

3 

Project  5 

3 

4 

Project  30 

3 

3 

Project  6 

3 

3 

Project  3 1 

4 

4 

Project  7 

4 

4 

Project  32 

4 

3  ; 

Project  8 

4 

4 

Project  33 

3 

3 

Project  9 

3 

3 

Project  34 

2 

3 

Project  10 

4 

4 

Project  35 

3 

3 

Project  1 1 

3 

3 

Project  36 

3 

3 

Project  12 

4 

4 

Project  37 

3 

3 

Project  13 

3 

3 

Project  38 

5 

5 

Project  14 

4 

4 

Project  39 

3 

3 

Project  15 

4 

4 

Project  40 

4 

4 

Project  16 

4 

3 

Project  41 

4 

4 

Project  17 

4 

4 

Project  42 

3 

3 

Project  18 

3 

3 

Project  43 

3 

3 

Project  19 

3 

3 

Project  44 

3 

4 

Project  20 

3 

3 

Project  45 

3 

3 

Project  21 

3 

4 

Project  46 

3 

3 

Project  22 

4 

4 

Project  47 

3 

3 

Project  23 

3 

3 

Project  48 

3 

3 

Project  24 

4 

4 

Project  49 

3 

3 

Project  25 

4 

4 

Project  50 

3 

3 

2 

When  comparing  the  R'  values  for  Model  #1  and  Model  #2,  accounting  for  the 
variance  in  the  contractor’s  overall  performance  rating  does  not  seem  to  be  significantly 
improved  by  combining  the  independent  variables  EM  and  TP.  The  same  holds  true 
when  the  two  models  are  validated;  mathematical  Model  #1  and  Model  #2  appear  to  have 
similar  prediction  power.  Using  the  50  project  evaluations  set  aside  for  model  validation, 
Model  #1  accurately  predicted  88%  of  the  contractor’s  overall  performance  ratings; 
Model  #2  accurately  predicted  90%.  Implications  of  these  results  will  be  discussed  in 
Chapter  5  of  this  research. 

Research  Question  3:  Is  the  contractor’s  overall  performance  rating  and  the  overall 
project  schedule  performance  related? 

Figure  17  is  a  conceptual  model  of  research  question  3.  Correlation  analysis  was 
conducted  on  the  165  project  evaluations  to  determine  if  a  relationship  exists  between  the 
contractor’s  overall  rating  and  the  overall  project  schedule  performance.  The  results  of 
this  correlation  analysis  were  a  Pearson’s  correlation  value  of  -0.195  and  a  significance 
value  of  0.006.  This  indicates  a  significant  relationship  between  the  contractor’s  overall 
performance  rating  and  the  overall  project  schedule  performance,  though  the  relationship 
is  small.  Implications  of  these  results  will  be  discussed  in  Chapter  5  of  this  research. 


Figure  17.  Conceptual  Model  of  Research  Question  3 
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Summary 

This  study  gathered  215  DD  Form  2626  construction  project  evaluations  to 
determine  if  any  of  the  performance  items  are  correlated  to  a  contractor’s  overall 
performance  rating.  This  data  were  evaluated  to  determine  if  the  33  performance  items 
reflected  their  respective  performance  elements.  A  principle  component  analysis  on  each 
performance  element  and  their  respective  performance  items  revealed  that  most 
performance  items  represented  their  respective  performance  elements;  however,  not  all 
performance  items  factored  together  as  represented  on  DD  Form  2626.  Aggregation  of 
the  performance  items  resulted  in  nine  performance  elements;  four  greater  than  the 
original  five  performance  elements  as  indicated  on  DD  Form  2626.  A  correlation 
analysis  was  then  conducted  on  these  nine  performance  elements  and  the  contractor’s 
overall  performance  rating  to  determine  if  the  performance  items  as  aggregated  into  the 
“factored”  performance  elements  are  an  indication  of  the  contractor’s  overall 
performance.  The  correlation  analysis  revealed  that  eight  of  the  nine  performance 
elements  had  a  significant  relationship  with  the  contractor’s  overall  rating.  The 
performance  element  that  did  not  have  a  relationship  with  the  contractor’s  overall 
performance  rating  was  Warranty  Response.  These  results  are  likely  explained  by  the 
fact  that  Warranty  Response  was  only  rated  for  66%  of  the  evaluated  construction 
projects.  Using  the  eight  performance  elements  that  had  a  significant  relationship  with 
the  contractor’s  overall  performance  rating,  a  linear  regression  analysis  was  conducted  to 
identify  if  these  performance  elements  were  indicators  of  the  contractor’s  overall 
performance.  When  the  step-wise  linear  regression  was  complete,  only  two  performance 
elements  were  predictive  of  the  contractor’s  overall  rating:  Effectiveness  of  Management 
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and  Timely  Performance.  Of  the  original  33  performance  items  used  to  rate  a 


contractor’s  performance,  only  twelve  have  significant  impact  on  how  the  contractor’s 
overall  performance  is  rated.  Finally,  one  measure  of  traditional  project  success  is 
schedule  performance  (Ling  &  Liu,  2004).  This  tradition  is  not  reflected  in  the 
contractor’s  overall  performance  rating  as  indicated  by  the  correlation  analysis  of  the 
contractor’s  overall  performance  rating  and  the  overall  project  schedule  performance. 
Table  12  is  a  summary  of  the  answers  to  each  of  the  research  questions.  The  implications 
of  the  results  summarized  here  will  be  discussed  in  detail  in  Chapter  5  of  this  research. 


j  Table  12.  Summary  of  Answers  to  the  Research  Questions  j 

Research  Question 

Answer 

Explanation 

1 

Do  the  DD  Form  2626 
performance  elements  reflect 
their  respective  performance 
items? 

Yes, 

however 

All  the  performance  items  factored  into  their  respective  performance  elements  with  the  exception  of 
Adequacy  of  As-Builts,  Coordination  and  Control  of  Subcontractors,  Review/Resolution  of 
Subcontractor's  Issues,  Implementation  of  Subcontracting  Plan,  Warranty  Response,  and  Correction  of 
Noted  Deficiences  (w.r.t.  Compliance  with  Labor  Standards).  Based  on  the  results  of  the  PCA,  the 
original  five  performance  elements  as  shown  on  DD  Form  2626  were  factored  into  the  following  nine 
performance  elements:  Quality  Control  (QC),  Adequacy  of  As-Builts  (AAB),  Effectiveness  of 
Management  (EM),  Management  of  Subcontractors  (MS),  Timely  Performance  (TP),  Warranty  Response 
(WR),  Compliance  with  Labor  Standards  (CLS),  Correction  of  CLS  Noted  Deficiencies  (CCLSND),  and 
Compliance  with  Safety  Standards  (CSS). 

2 

Do  the  performance  items  as 
they  are  appropriately 
aggregated  together  into 
performance  elements 
predict  the  contractor’s 
overall  performance  rating? 

Yes, 

however 

A  correlational  analysis  was  used  to  determine  the  relationship  between  the  contractor's  overall 
performance  rating  and  each  factored  performance  element.  Seven  of  the  nine  factored  performance 
elements  are  significantly  related  to  the  contractor's  overall  performance  rating.  However,  a  linear 
regression  analysis  revealed  that  only  two  of  the  seven  factored  performance  elements  are  predictive  of  a 
contractor's  overall  performance  rating. 

3 

Is  the  contractor’s  overall 
performance  rating  and  the 
overall  project  schedule 
performance  related? 

Yes, 

however 

A  significant  relationship  exists  between  the  contractor’s  overall  performance  rating  and  the  overall 
project  schedule  performance,  though  the  relationship  is  small. 
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V.  Conclusions  and  Recommendations 


Chapter  Overview 

This  chapter  summarizes  the  results  and  answers  the  questions  posed  by  each 
research  question.  Where  Chapter  4  presented  the  results  of  each  statistical  analysis 
conducted  on  the  data  provided  from  DD  Form  2626,  this  chapter  will  discuss  the 
significance  and  limitations  of  these.  Finally,  this  chapter  will  recommend  actions  that 
should  be  taken  as  a  result  of  this  research  and  recommend  future  areas  of  research. 

Problem  Statement 

The  objective  of  this  research  was  to  analyze  DD  Form  2626  -  the  USACE’s 
method  for  evaluating  construction  contractors — in  order  to  determine  if  their  process  is 
reliable,  valid,  and  appropriate  for  implementation  by  the  USAF  civil  engineers  when 
evaluating  and  rating  contractor  performance.  The  research  analyzed  DD  Form  2626  to 
detennine  if  any  of  the  performance  items  are  correlated  to  a  contractor’s  overall 
performance  rating.  The  research  determined  if  a  relationship  exists  between  the  project 
schedule  performance  and  the  contractor’s  overall  performance. 

Research  Questions 

1.  Do  the  DD  Form  2626  performance  elements  reflect  their  respective 

performance  items? 

To  test  if  the  five  performance  elements  provided  on  DD  Form  2626  are 
represented  by  their  respective  performance  items,  a  principle  component  analysis  was 
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conducted.  The  results  of  the  principle  component  analysis  revealed  that  the  performance 
elements  are  not  completely  reflective  of  the  performance  items  as  they  are  arranged  on 
DD  Form  2626.  Though  most  performance  items  factored  together  with  their  respective 
performance  element,  a  few  performance  items  factored  separately.  Those  performance 
items  were  Storage  of  Materials,  Adequacy  of  As-Builts,  Use  of  Specified  Materials, 
Coordination  and  Control  of  Subcontractors,  Review/Resolution  of  Subcontractor’s 
Issues,  Implementation  of  Subcontracting  Plan,  Warrantee  Response,  and  Correction  of 
Noted  Deficiencies  (under  Compliance  with  Labor  Standards).  It  is  possible  the  reason 
why  these  particular  performance  items  do  not  factor  with  their  respective  performance 
items  because  only  72%  of  the  contractor  evaluations  were  rated  for  Adequacy  of  As- 
Builts,  67%  were  rated  for  Warranty  Response  and  only  82%  were  rated  for  Correction  of 
Compliance  with  Labor  Standard  Noted  Deficiencies.  Without  access  to  the  individuals 
who  completed  these  contractor  evaluation  forms,  one  can  only  speculate  why  these 
performance  items  were  not  applicable  for  rating.  For  instance,  it  is  possible  that  only 
72%  of  the  projects  evaluated  required  as-builts  or  that  when  the  DD  Form  2626  was 
completed  for  a  particular  contractor,  they  had  not  submitted  the  as-builts  yet  for  review. 
As  indicated  by  the  local  USACE  resident  engineer,  this  can  sometimes  be  a  problem 
with  contractors.  While  the  contractor  is  required  by  contract  to  submit  as-builts  three 
months  prior  to  contract  completion,  typically  the  as-builts  are  not  turned  over  to  the 
USACE  office  until  sometime  after  the  construction  has  been  completed. 

The  same  holds  true  for  Warranty  Response  which  was  only  applicable  for  67% 
of  the  contractor  evaluations.  However,  one  should  question  how  appropriate  it  is  to  rate 
a  contractor’s  warranty  response  immediately  after  the  completion  of  construction.  A 
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contractor’s  response  to  warranty  items  would  only  begin  once  the  construction  project  is 
complete  and  depending  on  the  length  of  the  warranty  can  be  a  period  of  months  to  two 
years  after  the  construction  is  complete.  If  a  contractor  performance  evaluation  is 
completed  on  a  contractor  once  the  construction  has  been  completed,  there  is  not 
sufficient  time  or  opportunity  to  fairly  evaluate  a  contractor’s  warranty  response. 

With  82%  of  the  project  evaluations  including  a  rating  for  Correction  of 
Compliance  with  Labor  Standard  Noted  Deficiencies,  one  might  question  why  this 
performance  item  did  not  factor  with  the  other  two  performance  items  listed  under 
Compliance  with  Labor  Standards.  The  answer  may  correspond  with  how  ‘Not 
Applicable’  ratings  were  scored  for  the  statistical  analysis  and  the  reason  why  a 
contractor’s  perfonnance  might  be  rated  as  not  applicable  for  correction  of  noted 
deficiencies.  Table  13  is  a  snapshot  of  all  project  evaluations  where  Correction  of 
Compliance  with  Labor  Standards  Noted  Deficiencies  was  rated  as  Not  Applicable. 
Notice  that  for  the  other  two  related  performance  items,  the  contractor’s  perfonnance 
rating  was  either  satisfactory,  above  average,  or  outstanding.  It  is  possible  that  a 
contractor  who  properly  completed  and  submitted  payrolls  and  who  complied  with  labor 
laws  and  regulation,  likely  did  not  have  any  noted  deficiencies  with  compliance  with 
labor  standards.  If  the  contractor  did  not  have  any  noted  deficiencies  to  correct,  his 
ability  to  correct  noted  deficiencies  could  therefore  not  be  rated.  Recall  from  Chapter  3, 
for  statistical  analysis,  the  qualitative  ratings  were  given  numeric  scores  as  follows:  Not 
Applicable  =  0,  Outstanding  =  5,  Above  Average  =  4,  Satisfactory  =  3,  Marginal  =  2,  and 
Unsatisfactory  =  1.  Quantitatively,  a  rating  of  zero  is  very  different  from  a  rating  of 
three,  four,  or  five.  Therefore,  during  the  principle  component  analysis,  it  appears  that 
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the  three  performance  item  scores  listed  under  Compliance  with  Labor  Standards  do  not 


relate,  where  in  fact,  taken  from  an  adjusted  perspective,  they  are  highly  related.  This 
statement  becomes  true  when  the  scores  are  reversed  coded  such  that  1)  a  score  of  zero 
means  that  there  are  not  deficiencies  to  correct,  therefore  performance  in  this  item  is 
close  to  outstanding,  and  2)  if  a  contractor  is  rated  high  in  correction  of  noted 
deficiencies,  then  his  rating  corresponds  to  his  performance  in  completing  payrolls  and 
compliance  with  labor  laws  and  regulations.  When  a  principle  component  analysis  was 
performed  on  the  reverse  coded  data,  all  three  performance  items  factored  under  one 
component,  and  therefore  are  reflective  of  the  performance  element  Compliance  with 
Labor  Standards.  Results  of  this  principle  component  analysis  are  available  in  Appendix 
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Table  13.  Compliance  with  Labor  Standards 

Project 

18. a* 

18.b** 

18.c*** 

Project 

18. a* 

18.b** 

18. c*** 

Project  1 

0 

4 

4 

Project  16 

0 

3 

3 

Project  2 

0 

4 

4 

Project  17 

0 

3 

3 

Project  3 

0 

3 

3 

Project  18 

0 

3 

3 

Project  4 

0 

4 

3 

Project  19 

0 

4 

4 

Project  5 

0 

4 

3 

Project  20 

0 

4 

4 

Project  6 

0 

3 

3 

Project  21 

0 

3 

3 

Project  7 

0 

3 

3 

Project  22 

0 

3 

3 

Project  8 

0 

4 

4 

Project  23 

0 

3 

3 

Project  9 

0 

3 

3 

Project  24 

0 

3 

3 

Project  10 

0 

3 

3 

Project  25 

0 

4 

4 

Project  1 1 

0 

3 

3 

Project  26 

0 

4 

5 

Project  12 

0 

3 

3 

Project  27 

0 

4 

5 

Project  13 

0 

3 

3 

Project  28 

0 

5 

5 

Project  14 

0 

4 

4 

Project  29 

0 

4 

4 

Project  15 

0 

4 

4 

Project  30 

0 

3 

3 

*  Correction  of  Noted  Deficiences 
**  Payrolls  Properly  Completed  and  Submitted 

***  Compliance  with  Labor  Laws  and  Regulations  with  Specific  Attention  to  the  Davis-Bacon  Act  and  EEO 
Requirements 
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When  a  principle  component  analysis  was  conducted  on  the  performance  items 
listed  under  Effectiveness  of  Management,  the  result  was  three  of  the  nine  performance 
items  to  factor  into  a  separate  component.  The  three  performance  items  that  factored 
separately  were  Coordination  and  Control  of  Subcontractors,  Review/Resolution  of 
Subcontractor’s  Issues,  and  Implementation  of  Subcontracting  Plan.  Because  only  these 
three  performance  items  relate  to  a  contractor’s  ability  to  manage  his  subcontractor(s),  it 
is  appropriate  that  these  performance  items  would  factor  together.  Theoretically,  it 
should  also  not  surprising  that  these  three  performance  items  would  not  factor  with  the 
other  six  performance  items  listed  under  Effectiveness  of  Management  since  only  65%  of 
the  contractor  evaluations  included  subcontractor  work  as  indicated  in  Block  8  on  DD 
Form  2626;  however,  closer  examination  of  the  data  revealed  an  item  of  interests.  While 
only  65%  of  the  contractor  evaluations  were  listed  has  having  subcontractor  work 
performed,  96%  of  the  contractor  evaluations  rated  the  contractor’s  ability  to  coordinate 
and  control  subcontractor(s),  93%  of  the  contractors  were  evaluated  on  their  ability  to 
review  and  resolve  subcontractor(s)  issues,  and  finally  83%  of  the  contractors  were 
evaluated  on  their  ability  to  implement  the  subcontracting  plan.  Table  14  is  a  list  of  all 
the  projects  that  were  identified  has  having  zero  subcontracted  work.  The  table  also 
includes  the  contractor’s  performance  rating  on  all  performance  items  relating  to  the 
management  of  subcontractors.  Theoretically,  if  a  contractor  has  no  subcontracted  work, 
then  his  ratings  for  the  following  performance  items  should  all  be  ‘not  applicable  (N/A): 
Coordination  and  Control  of  Subcontractors,  Review/Resolution  of  Subcontractor’s 
Issues,  and  Implementation  of  Subcontracting  Plan;  however,  for  58  projects  listed  as 
having  no  subcontracted  work,  only  six  meet  this  theoretical  criterion  (see  the  shaded 
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boxes  in  Table  14).  Either  only  six  projects  of  the  total  165  projects  used  in  this  analysis 
did  not  have  subcontracted  work  performed  and  therefore  Block  9  on  DD  Form  2626 
includes  incorrect  data  for  52  projects;  or,  52  of  the  projects  were  inappropriately 
evaluated  for  subcontractor  performance,  suggesting  that  the  evaluator  did  not  pay  careful 
attention  to  the  performance  items  when  rating  the  contractor’s  performance.  If  the  latter, 
then  the  question  arises:  how  many  other  performance  items  were  incorrectly  rated  when 
evaluating  the  contractor’s  performance  and  what  effect  does  this  have  on  the 
contractor’s  overall  rating?  These  questions  cannot  be  answered  by  this  research  and 
therefore  impact  the  overall  reliability  of  the  DD  Form  2626. 

The  overall  result  of  the  principle  component  analysis  was  a  reduction  of  33 
independent  variables  (the  performance  items)  into  the  following  nine  independent 
variables:  Quality  Control  1  (QC1),  Quality  Control  2  (QC2),  Effectiveness  of 
Management  (EM),  Management  of  Subcontractors  (MS),  Timely  Performance  (TP), 
Warranty  Response  (WR),  Compliance  with  Labor  Standards  (CLS),  Correction  of  CLS 
Noted  Deficiencies  (CCLSND),  and  Compliance  with  Safety  Standards  (CSS). 
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j  Table  14.  Contractor  Performance  Rating  with  0%  Subcontracted  Work  | 

Project 

16.c* 

16.h** 

jg  j*** 

Project 

16.c* 

16.h** 

j*** 

Project 

16.c* 

16.h** 

Project  1 

4 

0 

0 

Project  20 

3 

3 

3 

Project  39 

3 

3 

3 

Project  2 

0 

0 

0 

Project  21 

3 

3 

3 

Project  40 

3 

3 

3 

Project  3 

0 

0 

0 

Project  22 

3 

3 

3 

Project  41 

3 

3 

3 

Project  4 

0 

0 

0 

Project  23 

3 

3 

3 

Project  42 

3 

3 

3 

Project  5 

0 

0 

0 

Project  24 

3 

3 

3 

Project  43 

3 

3 

3 

Project  6 

0 

0 

0 

Project  25 

3 

3 

3 

Project  44 

3 

3 

3 

Project  7 

0 

0 

0 

Project  26 

3 

3 

3 

Project  45 

3 

3 

3 

Project  8 

3 

3 

0 

Project  27 

3 

3 

3 

Project  46 

4 

3 

3 

Project  9 

3 

3 

0 

Project  28 

3 

3 

3 

Project  47 

2 

3 

3 

Project  10 

3 

3 

0 

Project  29 

3 

3 

3 

Project  48 

3 

3 

3 

Project  1 1 

3 

3 

0 

Project  30 

3 

3 

3 

Project  49 

3 

3 

3 

Project  12 

3 

3 

0 

Project  3 1 

3 

3 

3 

Project  50 

4 

3 

3 

Project  13 

3 

4 

0 

Project  32 

3 

3 

3 

Project  5 1 

4 

3 

3 

Project  14 

4 

4 

0 

Project  33 

3 

3 

3 

Project  52 

4 

4 

3 

Project  15 

4 

5 

0 

Project  34 

3 

3 

3 

Project  53 

4 

4 

3 

Project  16 

2 

3 

2 

Project  35 

3 

3 

3 

Project  54 

3 

4 

4 

Project  17 

2 

3 

3 

Project  36 

3 

3 

3 

Project  55 

4 

4 

4 

Project  18 

2 

3 

3 

Project  37 

3 

3 

3 

Project  56 

4 

5 

4 

Project  19 

2 

3 

3 

Project  38 

3 

3 

3 

Project  57 

4 

5 

4 

*  Coordination  and  Control  of  Subcontractor(s) 

Project  58 

5 

5 

4 

Review/Resolution  of  Subcontractor's  Issues 
*  Implementation  of  Subcontracting  Plan 


2.  Do  the  performance  items  as  they  are  appropriately  aggregated  into 

performance  elements  predict  the  contractor’s  overall  performance  rating? 

These  nine  acceptable  factored  performance  elements  were  used  as  independent 
variables  to  analyze  research  question  number  two.  Using  SPSS  version  16.0,  a 
correlation  analysis  was  conducted  to  determine  if  a  relationship  exists  between  the  nine 
performance  elements  derived  from  the  principle  component  analysis  and  the  contractor’s 
overall  performance  rating.  Eight  of  the  nine  performance  elements  analyzed  have  a 
significant  relationship  with  the  contractor’s  overall  performance  rating.  Effectiveness  of 
Management  has  the  strongest  correlation  to  the  contractor’s  overall  performance  rating, 
whereas,  Warranty  Response  has  no  relationship  to  the  contractor’s  overall  performance 
rating.  Correlation  analysis  only  considers  the  relationship  between  each  individual 
performance  element  and  the  contractor’s  overall  perfonnance  rating.  The  next  step  in 
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this  analysis  was  to  conducted  a  step-wise  linear  regression  in  order  to  understand  how 
the  performance  elements  interact  with  each  other  to  affect  the  contractor’s  overall 
performance  rating.  A  step-wise  linear  regression  was  conducted  on  those  performance 
elements  with  a  significant  relationship  to  the  contractor’s  overall  performance  rating  to 
determine  if  in  fact  the  contractor’s  overall  performance  rating  is  a  composite  score  of  the 
measured  performance  items  as  aggregated  into  performance  elements.  The  step-wise 
linear  regression  resulted  in  two  mathematical  models  for  predicting  a  contractor’s 
overall  performance  rating.  The  first  model  only  includes  Effectiveness  of  Management 
(EM)  as  an  independent  variable  with  an  R“  value  of  0.755.  The  second  model  includes 
both  Effectiveness  of  Management  and  Timely  Performance  (TP)  as  independent 
variables  with  an  R"  value  of  0.766  respectively.  When  comparing  the  R  values  for  each 
model,  the  contractor’s  overall  performance  rating  does  not  seem  to  be  significantly 
improved  by  combining  the  independent  variables  EM  and  TP.  The  same  holds  true 
when  the  two  models  are  validated;  mathematical  Model  #1  and  Model  #2  appear  to  have 
almost  the  same  prediction  power.  Using  the  50  project  evaluations  set  aside  for  model 
validation,  Model  #1  accurately  predicted  88%  of  the  contractor’s  overall  performance 
ratings;  Model  #2  accurately  predicted  90%. 

3.  Is  the  contractor’s  overall  performance  rating  and  the  overall  project 

schedule  performance  related? 

Correlation  analysis  was  conducted  on  the  165  project  evaluations  to  determine  if 
a  relationship  exists  between  the  contractor’s  overall  rating  and  the  overall  project 
schedule  performance.  The  results  of  this  correlation  analysis  were  a  Pearson’s 
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correlation  value  of  -0.195  and  a  significance  value  of  0.006.  There  is  a  significant 
relationship  between  the  contractor’s  overall  performance  rating  and  the  overall  project 
schedule  performance,  though  the  relationship  is  small.  This  result  reveals  the  overall 
project  schedule  performance  has  little  impact  on  the  contractor’s  overall  rating.  This  is 
contradictory  to  one  of  the  traditional  measures  of  a  contractor’s  performance;  quality, 
schedule,  and  cost  (Ling  &  Liu,  2004). 

Considering  that  the  quality  of  the  project  is  difficult  to  capture  immediately  after 
construction  is  completed,  and  the  cost  of  the  project  is  fixed,  then  all  that  remains  for  a 
project  manager  to  evaluate  is  a  contractor’s  schedule  perfonnance.  However,  as  the 
results  from  the  correlation  analysis  indicate,  the  project  managers  for  the  USACE  seem 
to  value  other  performance  elements  when  rating  the  overall  performance  of  the 
contractor.  The  results  from  the  correlation  analysis  and  linear  regression  from  research 
question  two  support  the  conclusion  that  project  managers  seem  to  value  the  contractor’s 
management  effectiveness  more  than  they  value  the  contractor’s  timely  performance. 
Further  correlation  analysis  of  each  Effectiveness  of  Management’s  perfonnance  items 
revealed  that  Effectiveness  of  Job-Site  Supervision  has  the  strongest  significant 
relationship  with  the  contractor’s  overall  performance  rating.  Table  15  summarizes  the 
results  of  the  correlation  analysis.  For  the  complete  results  of  the  correlation  analysis  see 
Appendix  P. 
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Table  15.  Correlation  Analysis  of  EM  Performance  Items  and 
Contractor's  Overall  Performance  Rating 

Contractor's  Overall 
Performance  Rating 

Cooperation  and  Responsiveness 

0.796** 

Management  of  Resources/Personnel 

0.758** 

Adequacy  of  Site  Clean-Up 

0.556** 

Effectiveness  of  Job-Site  Supervision 

0.801** 

Compliance  with  Laws  and  Regulations 

0.485** 

Professional  Conduct 

0.787** 

**  Correlation  is  significant  at  the  0.01  level 


Not  surprisingly  then  a  linear  regression  resulted  in  four  mathematical  prediction 
models,  where  Effectiveness  of  Job-Site  Supervision  as  an  individual  independent 
variable  was  predictive  of  a  contractor’s  overall  performance  rating  with  an  R  value  of 
0.641.  Recall  from  Chapter  4,  Effectiveness  of  Management  as  an  independent  variable 
is  an  average  score  of  the  ratings  given  to  each  of  the  related  performance  elements.  The 
mathematical  model  that  includes  only  Effectiveness  of  Management  as  an  independent 
variable  has  an  R2  value  of  0.755.  Therefore,  if  a  project  manager  wanted  to  predict  a 
contractor’s  overall  performance  rating,  he  could  theoretically  use  only  the  rating  of  a 
contractor’s  job-site  supervision  effectiveness  and  attain  approximately  the  same  results 
as  he  would  if  he  were  to  score  the  six  performance  items  that  are  reflective  of 
Effectiveness  of  Management,  average  those  scores,  and  then  apply  mathematical  model 
#1  described  in  Chapter  4.  The  R"  value  for  the  mathematical  prediction  model  increased 
to  0.760  when  ratings  for  performance  items  Professional  Conduct,  Compliance  with 
Laws  and  Regulations,  and  Professional  Conduct  were  added  to  the  independent  variable 
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Effectiveness  of  Job-Site  Supervision.  Results  of  the  step-wise  linear  regression  on  the 
Effectiveness  of  Management’s  performance  items  are  located  in  Appendix  Q. 

Significance  of  Research 

In  2007,  the  AF  Civil  Engineer,  Major  General  Del  Eulberg,  identified  35 
initiatives  in  an  effort  to  transform  the  civil  engineering  career  field  into  a  more  efficient 
and  effective  enterprise  (Eulberg,  2007).  One  such  initiative  is  Transformation  Project 
A-4.  The  purpose  of  this  initiative  is  to  strengthen  the  USAF  rating  system  of  contractor 
performance,  to  standardize  performance  criteria  and  eliminate  inconsistencies  between 
USAF  MAJCOMs  and  bases,  and  to  ensure  best  value  is  achieved  for  the  government  by 
establishing  a  system  of  rewards  and  penalties  for  good  and  bad  contractor  performance 
respectively.  By  studying  the  current  USAF  source  selection  practices,  this  research 
identified  the  limitations  of  the  selection  methods,  supporting  the  requirement  for  a  more 
rigorous  USAF  contractor  performance  rating  system  that  can  be  used  to  predict  future 
contractor  performance  as  requested  by  the  USAF  Civil  Engineer,  General  Del  Eulberg. 
Rather  than  inventing  a  new  method  for  evaluating  and  predicting  contractor 
performance,  this  research  examined  the  current  contractor  evaluation  employed  by  the 
USACE.  The  motivation  of  this  research  was  to  determine  if  the  construction 
performance  evaluation  process  used  by  the  USACE  is  reliable  and  valid,  and  if  so,  can  it 
be  adopted  by  the  USAF  civil  engineers  as  a  method  for  rating  and  predicting  contractor 
performance. 

The  research  determined  that  the  DD  Form  2626,  USACE’s  method  for 
evaluating  contractor  performance,  is  not  an  entirely  reliable  or  valid  process.  First,  not 
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all  of  the  performance  items  listed  on  the  DD  Form  2626  are  reflective  of  their  respective 
performance  elements.  This  means  that  project  managers  are  rating  performance  items 
that  don’t  actually  measure  the  performance  element  that  they  represent.  In  cases  such  as 
these,  either  the  performance  items  need  to  be  listed  into  a  separate  performance  element 
or  they  need  to  be  removed  from  the  evaluation  form  altogether. 

Second,  the  contractor’s  overall  performance  rating  is  not  a  composite  score  of 
the  33  performance  item  ratings.  As  indicated  by  the  correlation  analysis  and  linear 
regression  performed  on  the  nine  factored  performance  elements,  eight  of  the  factored 
performance  elements  had  a  significant  relationship  with  the  contractor’s  overall 
performance  rating;  and  of  those  eight,  only  two  factored  performance  elements, 
Effectiveness  of  Management  and  Timely  Performance,  are  strongly  indicative  of  a 
contractor’s  overall  performance.  This  result  was  confirmed  when  determining  the 
relationship  between  the  contractor’s  overall  performance  rating  and  the  overall  project 
schedule  performance.  The  contractor’s  inability  to  complete  a  construction  project 
according  to  the  revised  contract  completion  date  had  little  effect  on  a  contractor’s 
overall  performance  rating.  Further  analysis  indicated  that  not  only  was  a  contractor’s 
ability  to  manage  effectively,  but  specifically,  the  contractor’s  effective  job -site 
supervision  was  most  indicative  of  the  contractor’s  overall  performance  rating. 
Ultimately,  this  means  that  on  average  within  the  165  sampled  DD  Form  2626s,  the 
personnel  that  are  completing  the  DD  Form  2626  considered  a  contractor’s  job-site 
supervision  effectiveness  more  over  than  traditional  measures  such  as  schedule. 

Third,  when  decision  makers  use  past  contractor  overall  performance  ratings  in 
their  source  selection  process,  they  are  not  selecting  contractors  on  their  ability  to  provide 
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quality  control,  nor  timely  performance;  instead,  they  are  selecting  contractors  that  have 
demonstrated  satisfactory  or  above  average  performance  in  their  ability  to  manage 
effectively. 

Fourth,  if  only  Effectiveness  of  Management  is  indicative  of  a  contractor’s  overall 
performance  rating,  is  it  necessary  to  rate  the  other  27  performance  items,  or  can  they  be 
eliminated  from  DD  Form  2626?  This  should  be  considered  if  revising  DD  Form  2626, 
especially  since  this  research  has  demonstrated  that  DD  Form  2626  is  not  always 
accurately  completed.  Take  for  instance  the  discussion  of  subcontractor  management. 
Even  though  only  65%  of  the  projects  included  subcontracted  work,  93%  of  the 
contractor’s  were  rated  on  their  ability  to  coordinate  and  control  their  subcontractor(s). 
Either  more  projects  included  subcontracted  work  or  the  evaluator  incorrectly  rated  the 
contractor  on  subcontractor  performance  items. 

This  leads  to  the  fifth  concern  with  DD  Form  2626  -  the  raters  and  the  rating 
system.  Not  only  could  a  rater  fail  to  pay  close  attention  to  the  performance  items  they 
are  rating,  but  their  rating  style  could  undermine  the  meaningfulness  of  a  measurement. 
There  is  no  USACE  instruction  that  defines  or  standardizes  the  meaning  of  “satisfactory” 
performance,  therefore,  a  more  lenient  rater  may  rate  a  contractor’s  overall  performance 
as  above  average  where  a  more  strict  rater  could  possible  rate  the  same  contractor’s 
performance  as  satisfactory  or  even  marginal.  This  has  further  implication  to  contractor 
selection.  Technically,  if  disparities  exist  between  raters’  definition  or  understanding  of 
satisfactory  performance,  then  a  decision  maker  on  a  source  selection  cannot  use  past 
performance  scores  as  a  reliable  measure  for  distinguishing  between  contractors.  In 
addition,  the  layout  of  DD  Form  2626  encourages  the  rater  to  rate  the  contractor’s  overall 
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performance  rating  first  and  then  rate  each  individual  performance  item.  For  this  reason, 
raters  may  feel  compelled  to  align  their  ratings  of  each  individual  performance  item  to  the 
contractor’s  overall  performance  rating.  And  finally,  because  contractors  have  the 
opportunity  to  review  the  DD  Form  2626  before  it  is  finalized,  raters  may  be  compelled 
to  evaluate  the  contractor  favorably  in  order  to  avoid  confrontation. 

While  DD  Form  2626  represents  a  good  starting  point  to  meet  General  Eulberg’s 
transformation  request,  the  process  and  DD  Form  2626  requires  standardized  instructions 
and  a  new  format  that  aligns  performance  items  and  performance  elements  into  a  more 
rigorous  and  straightforward  system  for  rating  contractor  performance.  A  revised  DD 
Form  2626  that  could  be  used  on  any  size  and/or  type  of  construction  project  could 
strengthen  the  USAF  system  for  rating  contractor  performance.  If  the  revised  DD  Form 
2626  was  complimented  with  a  standardized  instruction,  then  this  tool  would  standardize 
performance  criteria  and  eliminate  inconsistencies  between  USAF  MAJCOMs  and  bases. 
Finally,  if  in  addition  to  an  instruction,  the  revised  DD  Form  2626  included  a  well 
defined  rating  system,  then  civil  engineers  across  the  Air  Force  could  ensure  best  value  is 
achieved  for  the  government  in  future  source  selections.  If  civil  engineers,  and  any  other 
decision  makers  for  that  matter,  have  a  clear  and  standardized  understanding  of 
‘satisfactory’  performance,  then  a  system  of  rewards  and  penalties  for  good  and  bad 
contractor  performance  could  be  fairly  and  appropriately  implemented  across  USAF 
MAJCOMs  and  bases.  This  has  important  implications  for  deployed  civil  engineers. 
Turnover  with  replacement  personnel  requires  imparting  a  large  amount  of  information  in 
a  very  limited  time  span.  Distinguishing  between  good  contractors  and  bad  contractors  is 
both  subjective  and  can  be  overlooked  during  turnover.  If  the  incumbent  civil  engineer 
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has  access  to  past  performance  evaluations  that  are  based  on  a  standardized  and  well 
defined  evaluation  system,  then  he  or  she  should  not  have  to  question  the  reliability  or 
validity  of  the  contractor’s  overall  performance  rating.  If  the  incumbent  civil  engineer 
could  immediately  distinguish  between  the  ‘good’  contractors  and  the  ‘bad’  contractors, 
the  civil  engineer  could  save  himself  valuable  time  and  energy,  while  at  the  same  time 
preserving  viable  and  limited  government  resources. 

Limitations  of  Research 

As  identified  in  Chapter  1,  one  of  the  major  limitations  of  this  research  is  the 
amount  of  available  data.  While  the  sample  size  is  large  enough  to  make  general 
conclusions  about  the  results  attained  the  various  statistical  analysis,  there  is  limited 
opportunity  to  validate  some  of  the  results,  specifically  the  linear  regression.  Using  a 
sample  size  of  50  limits  the  ability  to  test  the  predictive  power  of  mathematical  models 
#1  and  #2.  In  addition,  the  problem  of  limited  sample  data  prevents  the  research  from 
validly  declaring  difference  between  mathematical  models  #1  and  #2. 

The  second  limitation  of  the  study  is  the  unavailability  of  source  selection  data. 
This  research  would  be  very  powerful  if  we  could  assess  a  contractor’s  performance  as  it 
compares  to  their  past  performance  ratings.  This  type  of  analysis  would  empirically 
validate  the  statement  that  “past  performance  is  predictive  of  future  performance.”  It 
would  support  the  use  of  past  performance  ratings  and  information  for  source  selections. 
This  research  was  only  able  to  assess  an  existing  contractor  evaluation  tool  and  make 
recommendations  for  strengthening  its  reliability  and  validity  as  both  an  evaluation  tool 
and  a  means  for  source  selection. 
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The  third  limitation  of  this  research  is  the  anonymity  of  the  raters.  As  mentioned 


earlier,  one  rater’s  definition  or  understanding  of  ‘satisfactory’  performance  may  differ 
from  another  rater’s.  Therefore,  it  is  not  entirely  appropriate  to  compare  the  ratings  of 
each  performance  evaluation  because  we  cannot  establish  a  ‘true’  score  for  satisfactory 
performance.  If  the  identify  of  each  rater  was  known,  then  a  rating  style  for  each  rater 
could  be  identified  and  the  data  could  be  adjusted  to  normalize  the  scores.  This  would 
then  produce  more  reliable  ratings  which  would  in  turn  increase  the  validity  of  each 
statistical  analysis. 

The  fourth  limitation  of  the  research  is  the  generalizability  of  the  results.  All  215 
data  records  were  attained  from  a  regional  USACE  district  office  and  therefore,  the 
results  attained  in  this  research  may  only  be  applicable  to  this  region’s  USACE  offices. 

Recommendations  for  Action 

The  following  actions  are  recommended  if  General  Eulberg  and  the  civil  engineer 
community  are  to  implement  the  use  of  DD  Form  2626  as  a  means  standardizing  and 
strengthening  the  USAF  contractor  performance  evaluation  system.  First,  DD  Form  2626 
needs  to  be  reformatted  to  align  performance  items  such  that  they  reflect  their  respective 
performance  elements.  For  instance,  all  items  relating  to  subcontractor  issues  should  be 
consolidated  under  a  new  performance  element  titled  Management  of  Subcontractors.  In 
addition,  some  performance  items  may  warrant  total  removal  from  the  evaluation 
process.  Specifically,  Storage  of  Materials,  Adequacy  of  As-Builts,  Use  of  Specified 
Material,  and  Warranty  Response  could  be  removed  from  the  evaluation  form  and  not 
have  an  effect  on  the  contractor’s  overall  performance  rating.  Finally,  Block  1 1  -  Overall 
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Rating  should  be  relocated  directly  after  Block  19  -  Compliance  with  Safety  Standards. 
Placing  the  Overall  Rating  at  the  end  of  the  performance  element  ratings  encourages  the 
rater  to  evaluate  specific  performance  items  before  determining  an  overall  performance 
rating  for  the  contractor.  A  recommended  revised  DD  Form  2626  is  located  in  Appendix 
R.  To  further  strengthen  the  overall  rating  and  to  ensure  it  reflects  the  contractor’s 
performance  in  each  of  the  performance  elements,  the  Overall  Rating  could  be  a  score 
that  is  computed  from  the  ratings  of  each  performance  elements.  As  was  done  for 
research  question  two,  a  score  for  each  performance  element  would  be  determined  by 
averaging  the  ratings  of  each  respective  performance  item.  Then  each  performance 
element  score  could  be  multiplied  by  a  predetermined  weight  factor,  and  then  the  sum  of 
the  weighted  performance  element  scores  would  become  the  contractor’s  overall 
performance  rating. 

The  second  action  for  recommendation  is  a  standardized  and  well  defined  rating 
system.  An  example  of  such  a  system  is  the  one  implemented  by  the  National  Security 
Personnel  System  (NSPS).  Under  the  personnel  management  system,  performance 
indicators  are  defined  for  the  rater.  Each  subordinate’s  performance  is  rated  on  a  scale 
from  1  to  5  where  1  =  unsatisfactory  performance,  3  =  satisfactory  performance,  and  5  = 
outstanding  performance.  If  a  rater  evaluates  his  subordinate  performance  as  satisfactory, 
or  a  3,  then  he  must  be  able  to  justify  that  rating  using  a  pre-established  performance 
indicator  check  list.  For  instance,  a  non-supervisor  technician  in  pay  band  1  must  meet 
the  following  performance  indicators  if  his  rater  evaluates  his  performance  as 
satisfactory:  “with  supervision,  effectively  completed  assigned  job  objective  and  work 
assignments;  ensured  completed  work  adhered  to  given  instructions  and  standards;  in 
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achieving  job  objectives  and  work  assignments  adhered  to  work/project  schedules  and 
prioritization  work  tasks;  adjusted  scheduled  activities  as  directed  to  achieve  desired 
results  (Department  of  Defense,  2009).”  This  type  of  standardized  rating  system  aligns 
individual’s  definition  of  performance  terms  and  prevents  raters  from  rater  acquiescence 
or  yeah-saying.  This  would  also  prevent  a  rater  from  “gaming”  the  evaluation  if  the 
contractor’s  overall  performance  rating  was  calculated  using  weighted  performance 
element  scores. 

The  third  recommended  action  is  the  development  of  an  Air  Force  Instruction 
(AFI)  that  clearly  explains  how  to  correctly  complete  DD  Form  2626.  The  only 
instruction  USACE  has  for  the  application  of  DD  Form  2626  is  ER  415-1-17  Contractor 
Performance  Evaluations.  This  document  only  directs  the  use  of  DD  Form  2626  instead 
of  SF  1420  as  required  by  Federal  Acquisition  Regulations.  The  AFI  would  include 
detailed  instructions  of  what  information  needs  to  be  included  in  each  block  of  DD  Form 
2626.  The  AFI  would  also  include  the  standardized  and  defined  evaluation  system 
mentioned  earlier.  Finally,  it  would  include  instructions  on  the  use  of  DD  Form  2626  in 
future  source  selections.  Specifically,  it  would  identify  how  DD  Form  2626  ratings 
would  be  use  to  reward  or  punish,  good  or  bad  (respectively)  contractor  performance. 

The  fourth  recommended  action  is  to  reduce  the  total  number  of  performance 
items  that  must  be  evaluated  for  each  contractor.  This  recommended  action  concerns 
survey  response  as  a  social  exchange.  The  Webster-Merriam  dictionary’s  definition  of 
survey  is  “to  query  (someone)  in  order  to  collect  data  for  the  analysis  of  some  aspect  of  a 
group  or  area;  to  view  or  consider  comprehensively  (survey,  2009).”  Essentially,  the  DD 
Form  2626  Construction  Contractor  Performance  Evaluation  is  a  survey  being  conducted 
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by  the  USACE  on  each  of  its  contractors.  The  survey  respondent  in  this  situation  is  the 


USACE  project  manager  who  has  the  responsibility  of  completing  DD  Form  2626.  As 
Don  Dillman,  author  of  Mail  and  Internet  Surveys  points  out,  “longer  questionnaires 
achieve  slightly  lower  response  rates,”  whereas,  “research  has  shown  that  respondent- 
friendly  questionnaires,  with  carefully  organized  questions  in  easy-to-answer  formats, 
can  improve  response  rate  (Dillman,  2007).”  Eliminating  performance  items  that  have 
little  to  no  reflection  or  impact  on  the  contractor’s  overall  rating  will  increase  the  chance 
that  the  respondents,  in  this  case  the  project  engineers,  will  take  more  time  to  answer 
each  survey  question,  therefore,  providing  a  better  overall  evaluation  of  the  contractors 
performance. 

Finally,  for  each  construction  project,  more  than  one  rater  should  be  required  to 
complete  a  DD  Form  2626  for  contractor  performance  evaluation.  Raters  are  inherently 
subjective  and  inconsistent,  even  with  a  standardized  evaluation  system.  By  including 
multiple  raters  into  the  overall  evaluation  process,  an  aggregated  overall  evaluation  team 
score,  will  help  to  eliminate  or  at  least  reduce  inconsistencies  and  subjectivity,  thus 
achieving  a  more  rigorous  evaluation  system  (Expert  Choice,  Inc.,  1998). 

Recommendations  for  Future  Research 

Several  topics  have  emerged  from  this  research  that  would  benefit  the  Air  Force 
Civil  Engineer  community. 

1.  An  Analysis  of  USACE  Contractor  Evaluation  and  Selection  Methods.  This 
analysis  would  replicate  the  study  of  this  research;  however,  it  would  include  data 
attained  from  all  USACE  district  offices  throughout  the  US. 
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2.  Evaluating  Performance  Indicators  for  Rating  Contractor  Performance.  This 
research  would  include  a  survey  of  subject  matter  experts  (SMEs)  for  valued 
performance  indicators  to  be  incorporated  into  a  new  contractor  evaluation  form 
or  revised  DD  Form  2626.  This  research  would  also  include  identification  of  an 
appropriate  weighting  system  to  be  used  in  a  multiple-attribute  utility  system  for 
rating  and  predicting  contractor  performance. 

3.  A  Case  Study  of  Contractor  Selection  and  Project  Performance.  This  study  would 
examine  the  execution  of  a  construction  project  from  bid  evaluation,  to  source 
selection,  and  finally  construction  close  out.  The  study  would  include  a  detailed 
investigation  of  contractor  selection  and  contractor  performance  evaluation 
methods  used  for  the  duration  of  the  construction  project.  To  strengthen  the 
statement  “past  performance  is  indicative  of  future  performance”,  a  follow-on 
study  would  examine  the  specific  contractor’s  performance  in  a  future  project. 

4.  A  Validation  of  Pre-Qualification  Prediction  Model  for  USAF  Contractor 
Selection.  This  research  would  test  the  contractor  selection  prediction  model 
developed  by  Wong.  This  would  include  a  pre-survey  of  selected  contractors, 
administered  prior  to  the  beginning  of  construction,  that  focuses  on  various 
qualifying  criteria  and  a  post  survey,  administered  after  construction  is  complete, 
that  evaluates  the  contractor’s  performance.  Using  this  data,  the  researcher  would 
have  the  opportunity  to  validate  Wong’s  contractor  performance  prediction  model 
(Wong,  2004). 
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Summary 

This  chapter  described  the  conclusions  made  from  the  results  of  this  research  as 
they  related  to  each  research  question.  The  significance  of  this  research  was  identified 
along  with  the  limitations  of  this  research.  Recommended  actions  as  a  result  of  this 
research  were  outlined  and  a  list  of  future  research  options  that  would  expand  on  General 
Eulberg’s  desire  for  a  strengthened  and  standardized  contractor  performance  evaluation 
system  was  included  in  this  chapter.  This  study  provides  support  for  the  implementation 
of  the  US  ACE’s  contractor  evaluation  process  as  a  means  of  standardizing  contractor 
performance  criteria  and  eliminating  inconsistencies  between  USAF  MAJCOMs  and 
bases.  If  the  recommendations  for  action  are  implemented,  the  Air  Force  Civil  Engineer 
community  could  ensure  best  value  is  achieved  for  the  government  through  the  use  of 
rewards  and  penalties  for  good  and  bad  contractor  performance. 
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Appendix  A:  DD  Form  2626 


FOR  OFFICIAL  USE  ONLY  (WHEN  COMPLETED) 


PERFORMANCE  EVALUATION 
(CONSTRUCTION) 


1.  CONTRACT  NUMBER 


2.  CEC  NUMBER 


IMPORTANT:  Be  sure  to  complete  Part  III  -  Evaluation  of  Performance  Elements  on  reverse. 


PART  I  -  GENERAL  CONTRACT  DATA 


3.  TYPE  OF  EVALUATION  (X  one) 
I  INTERIM  (List percentage  


%) 


FINAL 


AMENDED 


4.  TERMINATED  FOR  DEFAULT 


5.  CONTRACTOR  (Name,  Address,  and  ZIP  Code) 


6. a.  PROCUREMENT  METHOD  (X  one) 

~1  SEALED  BID  I  I  NEGOTIATED 


b.  TYPE  OF  CONTRACT  (X  one) 
FIRM  FIXED  PRICE  f 
OTHER  (Specify) 


COST  REIMBURSEMENT 


7.  DESCRIPTION  AND  LOCATION  OF  WORK 


8.  TYPE  AND  PERCENT  OF  SUBCONTRACTING 


9.  FISCAL  DATA 


AMOUNT  OF  BASIC 
CONTRACT 


b.  TOTAL  AMOUNT  OF 
MODIFICATIONS 


c.  LIQUIDATED 

DAMAGES  ASSESSED 

$ 


d.  NET  AMOUNT  PAID 
CONTRACTOR 

$ 


10.  SIGNIFICANT 
DATES 


DATE  OF  AWARD 


b.  ORIGINAL  CONTRACT 
COMPLETION  DATE 


c.  REVISED  CONTRACT 
COMPLETION  DATE 


d.  DATE  WORK 
ACCEPTED 


PART  II  -  PERFORMANCE  EVALUATION  OF  CONTRACTOR 


1 1 .  OVERALL  RATING  (X  appropriate  block) 


OUTSTANDING 

ABOVE 

AVERAGE 

SATISFACTORY 

MARGINAL 

UNSATISFACTORY  (Explain 
in  Item  20  on  reverse) 


12.  EVALUATED  BY 


a.  ORGANIZATION  (Name  and  Address  (Include  ZIP  Code)) 


b.  TELEPHONE  NUMBER  (Include  Area 
Code) 


c.  NAME  AND  TITLE 


d.  SIGNATURE 


13.  EVALUATION  REVIEWED  BY 


a.  ORGANIZATION  (Name  and  Address  (Include  ZIP  Code)) 


b.  TELEPHONE  NUMBER  (Include  Area 
Code) 


c.  NAME  AND  TITLE 


d.  SIGNATURE 


14.  AGENCY  USE  (Distribution,  etc.) 


DD  FORM  2626,  JUN  94  (EG) 


EXCEPTION  TO  SF  1420  APPROVED  BY  GSA/IRMS  6-94 

Designed  using  Perform  Pro,  WHS/DIOR,  Sep  96 
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FOR  OFFICIAL  USE  ONLY  (WHEN  COMPLETED) 
PART  III  -  EVALUATION  OF  PERFORMANCE  ELEMENTS 


N/A  =  NOT  APPLICABLE  O  =  OUTSTANDING  A  =  ABOVE  AVERAGE  S  =  SATISFACTORY  M  =  MARGINAL  U  =  UNSATISFACTORY 


15.  QUALITY  CONTROL 

N/A 

0 

A 

S 

M 

U 

16.  EFFECTIVENESS  OF  MANAGEMENT 

N/A 

0 

A 

S 

M 

U 

a.  QUALITY  OF  WORKMANSHIP 

a.  COOPERATION  AND 

b.  ADEQUACY  OF  THE  CQC  PLAN 

b.  MANAGEMENT  OF  RESOURCES/ 
PERSONNEL 

c.  IMPLEMENTATION  OF  THE  CQC 
PLAN 

c.  COORDINATION  AND  CONTROL  OF 
SUBCONTRACTOR(S) 

d.  QUALITY  OF  QC 

DOCUMENTATION 

d.  ADEQUACY  OF  SITE  CLEAN-UP 

e.  STORAGE  OF  MATERIALS 

e.  EFFECTIVENESS  OF  JOB-SITE 
SUPERVISION 

f.  ADEQUACY  OF  MATERIALS 

g.  ADEQUACY  OF  SUBMITTALS 

f.  COMPLIANCE  WITH  LAWS  AND 
REGULATIONS 

h.  ADEQUACY  OF  QC  TESTING 

i.  ADEQUACY  OF  AS-BUILTS 

g.  PROFESSIONAL  CONDUCT 

j.  USE  OF  SPECIFIED  MATERIALS 

h.  REVIEW/RESOLUTION  OF 
SUBCONTRACTOR’S  ISSUES 

k.  IDENTIFICATION/CORRECTION  OF 
DEFICIENT  WORK  IN  A  TIMELY 
MANNER 

i.  IMPLEMENTATION  OF 
SUBCONTRACTING  PLAN 

17.  TIMELY  PERFORMANCE 

18.  COMPLIANCE  WITH  LABOR 
STANDARDS 

a.  ADEQUACY  OF  INITIAL  PROGRESS 
SCHEDULE 

a.  CORRECTION  OF  NOTED 

b.  ADHERENCE  TO  APPROVED 
SCHEDULE 

b.  PAYROLLS  PROPERLY  COMPLETED 
AND  SUBMITTED 

c.  RESOLUTION  OF  DELAYS 

c.  COMPLIANCE  WITH  LABOR  LAWS 

AND  REGULATIONS  WITH  SPECIFIC 
ATTENTION  TO  THE  DAVIS-BACON 

ACT  AND  EEO  REQUIREMENTS 

d.  SUBMISSION  OF  REQUIRED 
DOCUMENTATION 

e.  COMPLETION  OF  PUNCHLIST 

ITEMS 

19.  COMPLIANCE  WITH  SAFETY 
STANDARDS 

f.  SUBMISSION  OF  UPDATED  AND 
REVISED  PROGRESS  SCHEDULES 

a.  ADEQUACY  OF  SAFETY  PLAN 

b.  IMPLEMENTATION  OF  SAFETY  PLAN 

g.  WARRANTY  RESPONSE 

c.  CORRECTION  OF  NOTED 

20.  REMARKS  (Explanation  of  unsatisfactory  evaluation  is  required.  Other  comments  are  optional.  Provide  facts  concerning  specific  events 
or  actions  to  justify  the  evaluation.  These  data  must  be  in  sufficient  detail  to  assist  contracting  officers  in  determining  the  contractor’s 
responsibility.  Continue  on  separate  sheet(s),  if  needed.) 
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Appendix  B:  SF  1420 


FOR  OFFICIAL  USE  ONLY 
(WHEN  COMPLETED) 


PERFORMANCE  EVALUATION  -  CONSTRUCTION  CONTRACTS 

1.  CONTRACT  NUMBER 

1 .  CONTRACTOR  (Name,  address  and  ZIP  code) 

3.  TYPE  ^ 
OF  ^ 

CONTRACT 
(Check)  w 

A.  ADVERTISED 

B.  NEGOTIATED 

FIRM 

1 — |  i — |  FIXED  ( — |  OTHER 

U  CPFF  I I  PRICE  1 1  (Specify) 

4.  COMPLEXITY  OF  WORK 

DIFFICULT  |  |  ROUTINE 

5.  DESCRIPTION  AND  LOCATION  OF  WORK 


6.  FISCAL 

DATA 

► 

a.  AMOUNT  OF  BASIC 

CONTRACT 

$ 

b. TOTAL  AMOUNT  OF 
MODIFICATION 

$ 

c.  LIQUIDATED  DAMAGES 
ASSESSED 

$ 

d.NET  AMOUNT  PAID 

CONTRACTOR 

$ 

7.  SIGNIFICANT 
DATES 

► 

a.  DATE  OF  AWARD 

b. ORIGINAL  CONTRACT 
COMPLETION  DATE 

c.  REVISED  CONTRACT 
COMPLETION  DATE 

d.DATE  WORK  ACCEPTED 

8.  TYPE  AND  EXTENT  OF  SUBCONTRACTING 


PART  II  -  PERFORMANCE  EVALUATION  OF  CONTRACT  ( Check  appropriate  box ) 


9.  PERFORMANCE  ELEMENTS 

OUTSTANDING 

SATISFACTORY 

UNSATISFACTORY 

a.  QUALITY  OF  WORK 

b. TIMELY  PERFORMANCE 

c.  EFFECTIVENESS  OF  MANAGEMENT 

d. COMPLIANCE  WITH  LABOR  STANDARDS 

e.  COMPLIANCE  WITH  SAFETY  STANDARDS 

1 0.  OVERALL  EVALUATION 

|  OUTSTANDING  [Explain  in  Item  13,  on  reverse)  |  |  SATISFACTORY  UNSATISFACTORY  (Explain  in  Item  14,  on  reverse! 


1 1 .  EVALUATED  BY 

a.  ORGANIZATION  (Type  or  print) 


b.  NAME  AND  TITLE  (Type  or  print) 

c.  SIGNATURE 

d.  DATE 

12.  EVALUATION  REVIEWED  BY 

a.  ORGANIZATION  (Type  or  print) 


b. 


NAME  AND  TITLE  (Type  or  print) 


c.  SIGNATURE 


d.  DATE 


AUTHORIZED  FOR  LOCAL  REPRODUCTION 
Previous  edition  is  usable 


FOR  OFFICIAL  USE  ONLY 

(When  completed) 


STANDARD  FORM  1420  (10-83) 

Prescribed  by  GSA 

FAR  (48  CFR)  53.236-1(b) 
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FOR  OFFICIAL  USE  ONLY 

1  3.  REMARKS  ON  OUTSTANDING  PERFORMANCE  -  AS  INDICATED  BY  THE  CONTRACTOR'S  PERFORMANCE  ON  THIS  CONTRACT.  IF  YOU  CONSIDER  THE 
CONTRACTOR  TO  BE  OUTSTANDING,  SET  FORTH  FACTUAL  DATA  SUPPORTING  THIS  OBSERVATION.  THESE  DATA  MUST  BE  IN  SUFFICIENT  DETAIL  TO  ASSIST 
CONTRACTING  OFFICERS  IN  SELECTING  CONTRACTORS  THAT  HAVE  DEMONSTRATED  OUTSTANDING  QUALITY  OF  WORK  AND  RELIABILITY.  (Continue  on  separate 
sheet,  if  needed.) 


14.  EXPLANATION  OF  UNSATISFACTORY  EVALUATION.  FOR  EACH  UNSATISFACTORY  ELEMENT,  PROVIDE  FACTS  CONCERNING  SPECIFIC  EVENTS  OR  ACTIONS  TO 
JUSTIFY  THE  EVALUATION  (e.g.,  extent  of  Government  inspection  required,  rework  required,  subcontracting,  cooperation  of  contractor,  quality  of  workmen  and  adequacy  of 
equipment).  THESE  DATA  MUST  BE  IN  SUFFICIENT  DETAIL  TO  ASSIST  CONTRACTING  OFFICERS  IN  DETERMINING  THE  CONTRACTOR’S  RESPONSIBILITY.  ( Continue 
on  separate  sheet,  if  needed.) 


FOR  OFFICIAL  USE  ONLY 

(When  completed) 


STANDARD  FORM  1420  (10-83)  BACK 
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Appendix  C:  Raw  DD  Form  2626  Data  for  Model  Validation 


Contract  Number 

%  of  Subcontracting 

Amount  of  Basic 
Contract 

Total  Amount  of 
Modifications 

Liquidated 

Damages 

Assessed 

Net  Amount  Paid 
Contractor 

Project  Cost 
Growth 

DACA2700C0074 

0% 

$542,500 

$299,905 

$0 

$5,742,905 

5.82 

DACA2700D000501 90 

90% 

$349,907 

$12,892 

$0 

$362,799 

0.00 

DACA2701 D00030004 

0% 

$202,470 

$107,103 

$0 

$304,873 

-0.02 

DACA2701 D00030066 

0% 

$191,121 

$1 ,534 

$0 

$181,351 

-0.06 

DACA2701 D00030082 

35% 

$326,054 

$94,941 

$0 

$328,846 

-0.22 

DACA2701 D00030083 

11% 

$318,241 

($741) 

$0 

$316,219 

0.00 

DACA2701 D00030084 

1% 

$778,672 

$16,451 

$0 

$789,616 

-0.01 

DACA2702C0015 

35% 

$10,263,000 

$354,122 

$0 

$10,617,122 

0.00 

DACA2703D00080002 

0% 

$367,683 

$0 

$0 

$367,683 

0.00 

DACA2703D00080008 

0% 

$746,520 

$0 

$0 

$746,520 

0.00 

DACA2703D00080009 

0% 

$422,659 

$0 

$0 

$422,659 

0.00 

DACA2703D00080032 

74% 

$189,255 

$0 

$0 

$189,255 

0.00 

DACA2703D00080034 

100% 

$586,028 

$0 

$0 

$586,028 

0.00 

DACA2703D00080037 

100% 

$296,876 

$0 

$0 

$296,876 

0.00 

DACA2703D00080043 

100% 

$337,981 

$0 

$0 

$337,981 

0.00 

DACA2703D00080049 

74% 

$418,500 

$0 

$0 

$418,500 

0.00 

D  AC  A2703 D0008006 1 

80% 

$428,638 

$0 

$0 

$428,638 

0.00 

DACA2703D00080063 

80% 

$477,433 

$0 

$0 

$477,433 

0.00 

DACA2703D00080077 

56% 

$471 ,453 

$0 

$0 

$471 ,453 

0.00 

DACA2703D00080078 

59% 

$389,671 

$0 

$0 

$389,671 

0.00 

DACA2703D00080083 

78% 

$503,479 

$0 

$0 

$503,479 

0.00 

DACW2701 C001 8 

26% 

$29,800,000 

$20,362,359 

$0 

$50,142,359 

0.00 

DACW2701 D0005001 9 

0% 

$243,086 

$7,053 

$0 

$250,040 

0.00 

DACW2701 D00050022 

0% 

$225,671 

$15,356 

$0 

$225,671 

-0.06 

DACW2701 D00050030 

0% 

$171,646 

$0 

$0 

$171,646 

0.00 

DACW2701 D00050062 

22% 

$519,998 

$83,254 

$0 

$551,354 

-0.09 

DACW2701 D00050063 

0% 

$83,929 

$39,575 

$0 

$123,505 

0.00 

DACW2701 D00050066 

0% 

$1,910,000 

($17,841) 

$0 

$1,892,159 

0.00 

DACW2701 D00050068 

0% 

$309,991 

$9,400 

$0 

$319,391 

0.00 

DACW2701 D00050071 

100% 

$217,526 

$165,483 

$0 

$383,009 

0.00 

DACW2701 D00050085 

0% 

$327,348 

$9,700 

$0 

$337,048 

0.00 

DACW2797D0021 0163 

72% 

$110,570 

$0 

$0 

$110,570 

0.00 

DACW2797D0021 0165 

80% 

$112,988 

$0 

$0 

$112,988 

0.00 

W91 2QR04C0009 

26% 

$586,995,000 

$33,605,000 

$0 

$597,826,812 

-0.04 

W912QR04C0013 

80% 

$9,305,000 

$302,691 

$0 

$9,607,691 

0.00 

W91 2QR04C0029 

22% 

$2,030,872 

($270) 

$0 

$1,970,489 

-0.03 

W91 2QR04C0033 

78% 

$587,510 

$4,502 

$0 

$592,012 

0.00 

W912QR04D001 50020 

0% 

$152,500 

$0 

$0 

$152,500 

0.00 

W912QR04D001 60005 

19% 

$191,982 

$12,021 

$0 

$204,003 

0.00 

W91 2QR04D001 6001 2 

0% 

$146,792 

$24,444 

$0 

$171,236 

0.00 

W91 2QR05C001 9 

93% 

$2,127,152 

$934,415 

$0 

$3,061,567 

0.00 

W91 2QR05D0007001 4 

0% 

$145,410 

($8,000) 

$0 

$137,310 

0.00 

W91 2QR06C001 9 

13% 

$2,439,928 

$134,316 

$0 

$2,424,468 

-0.06 

W91 2QR06C0031 

74% 

$25,300,000 

$1 ,629,023 

$0 

$25,998,734 

-0.03 

W91 2QR06C0052 

62% 

$3,852,308 

$384,665 

$0 

$4,236,973 

0.00 

W91 2QR06C0059 

20% 

$1,701,596 

$0 

$0 

$1 ,677,842 

-0.01 

W91 2QR06D00080006 

85% 

$203,500 

$0 

$0 

$203,000 

0.00 

W91 2QR06D0008001 5 

82% 

$597,989 

$0 

$0 

$597,989 

0.00 

W91 2QR06D00080023 

80% 

$1,429,134 

$1,881 

$0 

$1,431,015 

0.00 

W91 2QR06D00080038 

72% 

$388,255 

$62,100 

$0 

$450,355 

0.00 
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Contract  Number 

Date  of  Award 

Original  Contract 
Completion  Date 

Revised  Contract 
Completion  Date 

Scheduled  Days 
Work 

Date  Work 
Accepted 

Actual  Days 
Work 

DACA2700C0074 

9/21/2000 

4/13/2002 

12/15/2003 

1180 

6/11/2003 

993 

DACA2700D000501 90 

8/7/2002 

12/5/2002 

3/5/2003 

210 

6/3/2004 

666 

DACA2701 D00030004 

6/27/2001 

10/19/2002 

6/16/2003 

719 

8/7/2003 

771 

DACA2701 D00030066 

9/19/2003 

1/17/2004 

7/9/2004 

294 

8/20/2004 

336 

DACA2701 D00030082 

3/18/2005 

6/16/2005 

6/16/2005 

90 

11/2/2005 

229 

DACA2701 D00030083 

3/21/2005 

1/15/2006 

1/15/2006 

300 

7/1/2005 

102 

DACA2701 D00030084 

3/30/2005 

9/26/2005 

9/26/2005 

180 

9/6/2006 

525 

DACA2702C001 5 

6/28/2002 

1/20/2004 

12/6/2004 

892 

12/6/2004 

892 

DACA2703D00080002 

9/27/2003 

5/1/2004 

5/1/2004 

217 

9/30/2004 

369 

DACA2703D00080008 

9/29/2003 

5/6/2004 

5/6/2004 

220 

4/30/2004 

214 

DACA2703D00080009 

9/29/2003 

2/11/2004 

5/11/2004 

225 

5/11/2004 

225 

DACA2703D00080032 

8/18/2004 

1/23/2005 

4/30/2005 

255 

4/30/2005 

255 

DACA2703D00080034 

8/25/2004 

3/8/2005 

3/8/2005 

195 

4/28/2005 

246 

DACA2703D00080037 

9/3/2004 

1/18/2005 

1/18/2005 

137 

11/18/2004 

76 

DACA2703D00080043 

9/16/2004 

6/27/2005 

6/27/2005 

284 

3/31/2005 

196 

DACA2703D00080049 

9/22/2004 

5/2/2005 

5/2/2005 

222 

12/31/2005 

465 

DACA2703D00080061 

4/28/2005 

5/8/2006 

5/8/2006 

375 

4/30/2006 

367 

DACA2703D00080063 

5/12/2005 

2/24/2006 

2/24/2006 

288 

12/31/2005 

233 

DACA2703D00080077 

9/29/2005 

10/22/2006 

10/22/2006 

388 

8/25/2006 

330 

DACA2703D00080078 

9/30/2005 

10/22/2006 

10/22/2006 

387 

10/22/2006 

387 

DACA2703D00080083 

12/27/2005 

11/20/2006 

11/20/2006 

328 

9/21/2006 

268 

DACW2701 C001 8 

5/3/2001 

5/18/2004 

6/27/2005 

1516 

12/19/2005 

1691 

DACW2701 D0005001 9 

5/24/2002 

10/25/2002 

5/31/2003 

372 

4/3/2003 

314 

D  AC  W 2701 D00050022 

7/24/2002 

10/13/2002 

10/13/2002 

81 

10/13/2002 

81 

D  AC  W 2701 D00050030 

9/27/2002 

2/18/2003 

2/18/2003 

144 

2/18/2003 

144 

D  AC  W 2701 D00050062 

4/30/2003 

10/1/2003 

10/30/2006 

1279 

9/19/2006 

1238 

DACW2701 D00050063 

5/2/2003 

8/18/2003 

8/18/2003 

108 

8/18/2003 

108 

D  AC  W 2701 D00050066 

5/21/2003 

9/21/2003 

9/21/2003 

123 

7/7/2004 

413 

D  AC  W 2701 D00050068 

5/22/2003 

1010/2003 

9/15/2004 

482 

3/28/2005 

676 

DACW2701 D00050071 

6/19/2003 

10/30/2003 

4/15/2004 

301 

4/15/2004 

301 

D  AC  W 2701 D00050085 

7/22/2003 

12/4/2003 

12/4/2003 

135 

11/3/2004 

470 

DACW2797D0021 01 63 

6/26/2001 

10/26/2001 

10/26/2001 

122 

10/18/2001 

114 

DACW2797D0021 01 65 

6/28/2001 

10/10/2001 

10/10/2001 

104 

9/28/2001 

92 

W91 2QR04C0009 

3/4/2004 

11/15/2004 

1/15/2006 

682 

1/15/2006 

682 

W912QR04C0013 

6/17/2004 

12/27/2005 

6/12/2006 

725 

6/12/2006 

725 

W91 2QR04C0029 

9/24/2004 

10/16/2005 

2/15/2006 

509 

2/15/2006 

509 

W91 2QR04C0033 

9/30/2004 

3/5/2005 

3/5/2005 

156 

4/21/2005 

203 

W912QR04D001 50020 

3/29/2005 

8/30/2005 

10/13/2005 

198 

10/6/2005 

191 

W912QR04D001 60005 

9/23/2004 

4/28/2005 

4/28/2005 

217 

4/25/2005 

214 

W91 2QR04D001 6001 2 

2/1/2005 

7/29/2005 

7/29/2005 

178 

10/29/2005 

270 

W912QR05C0019 

6/30/2005 

4/8/2006 

4/13/2006 

287 

4/13/2006 

287 

W91 2QR05D0007001 4 

8/16/2006 

9/15/2006 

9/15/2006 

30 

2/14/2008 

547 

W912QR06C0019 

3/29/2006 

5/10/2007 

6/29/2007 

457 

6/26/2007 

454 

W912QR06C0031 

6/8/2006 

2/7/2008 

4/12/2008 

674 

4/4/2008 

666 

W91 2QR06C0052 

9/28/2006 

10/23/2007 

2/20/2008 

510 

2/20/2008 

510 

W91 2QR06C0059 

9/29/2006 

10/2/2007 

10/2/2007 

368 

10/2/2007 

368 

W91 2QR06D00080006 

6/29/2006 

11/9/2006 

11/9/2006 

133 

1/17/2007 

202 

W91 2QR06D0008001 5 

8/24/2006 

5/4/2007 

5/4/2007 

253 

5/1/2007 

250 

W91 2QR06D00080023 

12/21/2006 

10/2/2007 

10/2/2007 

285 

4/26/2007 

126 

W91 2QR06D00080038 

6/25/2007 

10/14/2007 

10/14/2007 

111 

9/5/2007 

72 
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Contract  Number 

Project  Schedule 
Growth 

Overall 

Rating 

15.a 

15.b 

15.c 

15.d 

15.e 

15.f 

15.g 

15.h 

15.i 

1 5-j 

15.k 

DACA2700C0074 

-16% 

3 

4 

3 

3 

3 

3 

4 

3 

3 

3 

4 

3 

DACA2700D000501 90 

21 7% 

3 

4 

3 

3 

3 

3 

3 

3 

3 

4 

3 

3 

DACA2701 D00030004 

7% 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

DACA2701 D00030066 

14% 

3 

4 

3 

3 

3 

3 

3 

3 

3 

3 

3 

4 

DACA2701 D00030082 

154% 

3 

3 

3 

3 

0 

3 

3 

0 

0 

0 

0 

4 

DACA2701 D00030083 

-66% 

3 

3 

3 

2 

2 

3 

4 

3 

0 

0 

3 

3 

DACA2701 D00030084 

192% 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

DACA2702C001 5 

0% 

3 

3 

3 

2 

2 

3 

3 

4 

3 

3 

3 

3 

DACA2703D00080002 

70% 

3 

4 

4 

3 

4 

3 

3 

3 

3 

0 

3 

3 

DACA2703D00080008 

-3% 

4 

4 

4 

4 

4 

3 

3 

3 

3 

3 

0 

3 

DACA2703D00080009 

0% 

4 

4 

4 

4 

4 

3 

3 

3 

3 

3 

0 

0 

DACA2703D00080032 

0% 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

DACA2703D00080034 

26% 

3 

4 

3 

3 

3 

3 

3 

3 

3 

0 

3 

3 

DACA2703D00080037 

-45% 

3 

4 

3 

3 

3 

3 

3 

3 

3 

0 

3 

3 

DACA2703D00080043 

-31% 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

DACA2703D00080049 

109% 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

DACA2703D00080061 

-2% 

4 

4 

4 

4 

4 

3 

3 

3 

3 

0 

3 

4 

DACA2703D00080063 

-19% 

4 

4 

4 

4 

4 

3 

3 

4 

3 

3 

3 

4 

DACA2703D00080077 

-15% 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

DACA2703D00080078 

0% 

3 

3 

3 

3 

3 

3 

3 

3 

3 

0 

3 

3 

DACA2703D00080083 

-18% 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

DACW2701 C001 8 

12% 

4 

4 

3 

3 

3 

4 

3 

3 

3 

3 

3 

3 

DACW2701 D0005001 9 

-16% 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

DACW2701 D00050022 

0% 

3 

3 

3 

3 

3 

0 

3 

3 

0 

4 

3 

3 

DACW2701 D00050030 

0% 

4 

3 

3 

4 

3 

0 

3 

3 

4 

0 

3 

3 

DACW2701 D00050062 

-3% 

3 

3 

3 

3 

3 

3 

4 

3 

0 

0 

4 

3 

DACW2701 D00050063 

0% 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

DACW2701 D00050066 

236% 

4 

4 

4 

5 

4 

0 

4 

4 

3 

0 

5 

4 

DACW2701 D00050068 

40% 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

DACW2701 D00050071 

0% 

3 

5 

3 

3 

3 

0 

4 

3 

3 

0 

4 

4 

DACW2701 D00050085 

248% 

4 

4 

3 

3 

3 

4 

4 

3 

3 

3 

4 

4 

DACW2797D0021 01 63 

-7% 

3 

3 

3 

3 

3 

3 

3 

3 

0 

0 

3 

3 

DACW2797D0021 01 65 

-12% 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

W91 2QR04C0009 

0% 

4 

4 

4 

3 

4 

3 

3 

3 

4 

4 

4 

4 

W91 2QR04C001 3 

0% 

4 

4 

5 

3 

3 

4 

4 

4 

4 

4 

4 

3 

W91 2QR04C0029 

0% 

2 

3 

3 

2 

2 

3 

3 

2 

3 

3 

3 

3 

W91 2QR04C0033 

30% 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

W912QR04D001 50020 

-4% 

4 

5 

3 

3 

3 

3 

5 

3 

4 

0 

4 

4 

W912QR04D001 60005 

-1% 

5 

5 

5 

4 

4 

4 

4 

4 

4 

4 

4 

4 

W91 2QR04D001 6001 2 

52% 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

W91 2QR05C001 9 

0% 

3 

4 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

W91 2QR05D0007001 4 

1723% 

3 

2 

3 

2 

2 

3 

3 

3 

0 

0 

3 

2 

W91 2QR06C001 9 

-1% 

4 

4 

3 

4 

4 

3 

3 

3 

3 

0 

3 

3 

W91 2QR06C0031 

-1% 

4 

4 

3 

3 

3 

4 

4 

4 

3 

3 

4 

4 

W91 2QR06C0052 

0% 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

W91 2QR06C0059 

0% 

4 

4 

3 

4 

3 

3 

3 

3 

3 

3 

3 

4 

W91 2QR06D00080006 

52% 

3 

5 

3 

3 

3 

3 

3 

3 

3 

0 

3 

3 

W91 2QR06D0008001 5 

-1% 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

W91 2QR06D00080023 

-56% 

4 

4 

3 

3 

3 

3 

0 

3 

3 

0 

0 

0 

W91 2QR06D00080038 

-35% 

4 

5 

3 

3 

3 

0 

4 

3 

0 

0 

4 

3 
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Contract  Number 

16.a 

16.b 

16.c 

16. d 

16.e 

16.f 

16. g 

16.h 

16.i 

17.a 

17.b 

17.c 

17.d 

17.e 

17.f 

17.fl 

DACA2700C0074 

4 

4 

3 

3 

4 

3 

4 

4 

3 

4 

3 

3 

3 

4 

3 

4 

DACA2700D000501 90 

4 

3 

3 

3 

3 

3 

3 

3 

3 

2 

2 

2 

3 

0 

0 

DACA2701 D00030004 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

0 

DACA2701 D00030066 

4 

2 

2 

3 

2 

3 

3 

3 

0 

0 

0 

3 

3 

3 

0 

3 

DACA2701 D00030082 

4 

3 

2 

3 

4 

3 

3 

2 

0 

0 

0 

0 

0 

3 

0 

3 

DACA2701 D00030083 

3 

2 

3 

3 

3 

3 

3 

0 

0 

0 

4 

0 

3 

3 

0 

0 

DACA2701 D00030084 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

0 

DACA2702C001 5 

2 

2 

2 

3 

3 

4 

4 

2 

4 

2 

2 

2 

2 

1 

3 

3 

DACA2703D00080002 

4 

4 

3 

3 

4 

4 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

DACA2703D00080008 

4 

4 

4 

4 

4 

3 

4 

4 

4 

3 

3 

3 

3 

3 

3 

3 

DACA2703D00080009 

4 

4 

4 

4 

4 

4 

4 

4 

4 

3 

3 

3 

3 

3 

3 

3 

DACA2703D00080032 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

DACA2703D00080034 

4 

4 

4 

4 

4 

4 

4 

4 

4 

3 

3 

3 

3 

3 

3 

3 

DACA2703D00080037 

4 

4 

3 

3 

3 

3 

3 

CO 

CO 

CO 

CO 

3 

CO 

3 

3 

3 

DACA2703D00080043 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

DACA2703D00080049 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

DACA2703D00080061 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

DACA2703D00080063 

4 

4 

4 

4 

4 

4 

4 

4 

4 

3 

3 

3 

3 

3 

3 

3 

DACA2703D00080077 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

DACA2703D00080078 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

DACA2703D00080083 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

DACW2701 C001 8 

4 

4 

4 

4 

4 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

0 

DACW2701 D0005001 9 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

0 

3 

DACW2701 D00050022 

2 

2 

3 

3 

3 

3 

3 

3 

3 

3 

3 

2 

3 

3 

3 

DACW2701 D00050030 

3 

3 

4 

n 

3 

3 

3 

3 

0 

0 

0 

3 

3 

0 

3 

DACW2701 D00050062 

2 

3 

4 

4 

3 

3 

4 

0 

0 

3 

2 

3 

3 

0 

2 

0 

DACW2701 D00050063 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

DACW2701 D00050066 

5 

4 

0 

0 

4 

4 

5 

0 

0 

3 

3 

4 

3 

3 

0 

0 

DACW2701 D00050068 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

0 

0 

DACW2701 D00050071 

4 

4 

3 

3 

3 

3 

3 

3 

0 

0 

0 

3 

3 

3 

0 

3 

DACW2701 D00050085 

4 

4 

4 

4 

4 

4 

4 

4 

3 

4 

4 

4 

3 

3 

4 

3 

DACW2797D0021 01 63 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

0 

3 

3 

0 

3 

DACW2797D0021 01 65 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

0 

3 

W91 2QR04C0009 

5 

4 

3 

3 

5 

4 

5 

4 

4 

4 

5 

5 

3 

4 

3 

0 

W91 2QR04C001 3 

4 

3 

3 

4 

3 

4 

5 

3 

3 

5 

3 

3 

3 

3 

3 

5 

W91 2QR04C0029 

2 

2 

1 

3 

3 

3 

3 

1 

0 

3 

2 

3 

2 

3 

2 

3 

W91 2QR04C0033 

3 

3 

3 

3 

3 

3 

3 

3 

0 

3 

3 

3 

3 

3 

3 

3 

W912QR04D001 50020 

4 

3 

0 

4 

4 

3 

4 

3 

3 

3 

3 

0 

2 

4 

0 

0 

W912QR04D001 60005 

5 

5 

5 

5 

5 

4 

5 

5 

3 

3 

4 

5 

4 

5 

3 

4 

W91 2QR04D001 6001 2 

3 

3 

3 

3 

3 

4 

3 

3 

3 

3 

3 

3 

3 

3 

3 

W91 2QR05C001 9 

4 

3 

4 

3 

3 

3 

4 

3 

3 

3 

4 

4 

3 

3 

3 

3 

W91 2QR05D0007001 4 

3 

3 

3 

2 

3 

0 

0 

3 

3 

3 

3 

3 

3 

3 

W91 2QR06C001 9 

4 

3 

3 

3 

4 

3 

4 

3 

3 

3 

3 

3 

3 

3 

3 

0 

W91 2QR06C0031 

4 

3 

4 

4 

4 

3 

4 

4 

3 

3 

4 

4 

3 

3 

3 

0 

W91 2QR06C0052 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

W91 2QR06C0059 

5 

3 

3 

4 

4 

3 

4 

3 

3 

3 

3 

4 

3 

4 

3 

3 

W91 2QR06D00080006 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

0 

3 

W91 2QR06D0008001 5 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

4 

3 

3 

4 

3 

4 

W91 2QR06D00080023 

4 

4 

4 

4 

4 

3 

4 

4 

3 

4 

4 

4 

3 

4 

0 

0 

W91 2QR06D00080038 

4 

4 

4 

4 

3 

3 

4 

3 

3 

3 

3 

0 

3 

4 

3 

0 
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Contract  Number 

18.a 

18.b 

18.c 

19.a 

19. b 

19.c 

DACA2700C0074 

0 

3 

4 

4 

3 

0 

DACA2700D000501 90 

0 

3 

3 

3 

3 

3 

DACA2701 D00030004 

3 

3 

3 

3 

3 

3 

DACA2701 D00030066 

3 

3 

3 

3 

3 

3 

DACA2701 D00030082 

3 

3 

3 

3 

3 

3 

DACA2701 D00030083 

0 

3 

3 

3 

3 

3 

DACA2701 D00030084 

3 

3 

3 

3 

3 

3 

DACA2702C0015 

3 

3 

3 

4 

3 

3 

DACA2703D00080002 

3 

3 

3 

3 

3 

3 

DACA2703D00080008 

3 

3 

3 

4 

4 

3 

DACA2703D00080009 

0 

3 

3 

4 

4 

0 

DACA2703D00080032 

3 

3 

3 

3 

3 

3 

DACA2703D00080034 

3 

3 

3 

3 

3 

3 

DACA2703D00080037 

3 

3 

3 

3 

3 

3 

DACA2703D00080043 

4 

4 

4 

4 

4 

4 

DACA2703D00080049 

3 

3 

3 

3 

3 

3 

DACA2703D00080061 

4 

4 

4 

4 

4 

4 

DACA2703D00080063 

3 

3 

3 

4 

4 

3 

DACA2703D00080077 

3 

3 

3 

3 

3 

3 

DACA2703D00080078 

3 

3 

3 

3 

3 

3 

DACA2703D00080083 

3 

3 

3 

3 

3 

3 

DACW2701 C001 8 

3 

3 

2 

3 

4 

4 

DACW2701 D00050019 

3 

3 

3 

3 

3 

0 

DACW2701 D00050022 

3 

3 

3 

3 

3 

3 

DACW2701 D00050030 

3 

3 

3 

3 

3 

3 

DACW2701 D00050062 

0 

3 

4 

3 

3 

0 

DACW2701 D00050063 

3 

3 

3 

3 

3 

3 

DACW2701 D00050066 

4 

3 

5 

4 

4 

5 

DACW2701 D00050068 

0 

3 

3 

3 

3 

3 

DACW2701 D00050071 

3 

3 

3 

3 

3 

3 

DACW2701 D00050085 

0 

3 

3 

4 

4 

0 

DACW2797D0021 01 63 

0 

3 

3 

3 

3 

0 

DACW2797D0021 01 65 

3 

3 

3 

3 

3 

0 

W91 2QR04C0009 

4 

3 

4 

4 

4 

4 

W91 2QR04C001 3 

4 

4 

4 

5 

5 

5 

W91 2QR04C0029 

3 

3 

3 

3 

2 

3 

W91 2QR04C0033 

3 

3 

3 

3 

3 

3 

W912QR04D001 50020 

4 

3 

3 

3 

3 

3 

W912QR04D001 60005 

3 

3 

3 

5 

5 

5 

W91 2QR04D001 60012 

3 

3 

3 

3 

3 

3 

W91 2QR05C001 9 

4 

4 

4 

3 

3 

4 

W91 2QR05D0007001 4 

0 

3 

3 

3 

3 

0 

W91 2QR06C001 9 

0 

3 

3 

3 

3 

3 

W912QR06C0031 

3 

3 

3 

4 

4 

4 

W91 2QR06C0052 

3 

3 

3 

3 

3 

3 

W912QR06C0059 

3 

3 

3 

3 

3 

4 

W91 2QR06D00080006 

3 

3 

3 

3 

3 

3 

W91 2QR06D0008001 5 

4 

3 

3 

3 

4 

4 

W91 2QR06D00080023 

0 

5 

3 

3 

4 

0 

W91 2QR06D00080038 

0 

3 

3 

3 

4 

0 
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Appendix  D:  Raw  DD  Form  2626  Data  for  Statistical  Analysis 


Contract  Number 

%  of  Subcontracting 

Amount  of  Basic 

Contract 

Total  Amount  of 

Modifications 

Liquidated 

Damages 

Assessed 

Net  Amount  Paid 

Contractor 

Project  Cost  Growth 

DACA2700D000500 1 9 

0.00 

$2,147,843 

$4,199,190 

$0 

$6,147,117 

-0.03 

DACA2700D00050 157 

0.90 

$599,998 

$9,622 

$0 

$588,598 

-0.03 

DACA270 1 D00080003 

0.90 

$1,121,184 

$45,301 

$0 

$1,166,485 

0.00 

DACA270 1 D00080040 

0.84 

$663,314 

$17,126 

$0 

$680,440 

0.00 

DACA2702C00 1 3 

0.80 

$11,202,000 

$687,621 

$0 

$11,889,621 

0.00 

DACA2703D0008007 1 

0.80 

$267,854 

$0 

$0 

$267,854 

0.00 

DACA2703D00080072 

0.73 

$401,408 

$0 

$0 

$401,408 

0.00 

DACA2703D00080079 

0.78 

$392,488 

$0 

$0 

$392,488 

0.00 

DACW2700C00 1 6 

0.23 

$8,291,350 

$195,455 

$0 

$5,754,437 

-0.32 

DACW2703C0025 

0.00 

$2,989,620 

$83,017 

$0 

$3,047,637 

-0.01 

W9 1 2QR04D00 1 600 1 1 

0.00 

$218,435 

$1,995 

$0 

$220,430 

0.00 

W912QR04D00 160008 

0.75 

$2,047,000 

$20,845 

$0 

$2,067,845 

0.00 

W9 1 2QR06C00 1 0 

0.00 

$13,813,700 

$0 

$0 

$13,502,154 

-0.02 

W9 1 2QR06C0044 

0.47 

$14,342,000 

$602,275 

$0 

$14,635,030 

-0.02 

W912QR07C0040 

0.40 

$1,450,144 

$49,438 

$0 

$1,485,032 

-0.01 

DACA270 1 D00090009 

0.54 

$4,120,759 

$56,060 

$0 

$4,135,050 

-0.01 

DACW270 1 D00050029 

0.00 

$377,948 

$0 

$0 

$377,948 

0.00 

W9 1 2QR04D00 150014 

0.50 

$495,685 

$98,909 

$0 

$551,640 

-0.07 

W9 1 2QR04D00 150014 

0.91 

$1,415,000 

$1,675 

$0 

$1,416,675 

0.00 

W912QR05C0013 

0.62 

$1,191,828 

$95,390 

$0 

$1,287,217 

0.00 

W9 1 2QR06D00080040 

0.25 

$123,758 

$0 

$0 

$123,758 

0.00 

W9 1 2QR06C0008 

0.51 

$8,449,300 

$367,823 

$0 

$8,752,466 

-0.01 

DACW270 1 D00050052 

0.00 

$109,006 

$1,317 

$0 

$110,324 

0.00 

DACW270 1 D00050078 

0.86 

$579,952 

$0 

$0 

$579,952 

0.00 

W912QR04D00 160027 

0.31 

$385,000 

$19,540 

$0 

$404,540 

0.00 

W9 1 2QR06D00080007 

1.00 

$258,964 

$359,075 

$0 

$618,039 

0.00 

W9 1 2QR06D00080032 

0.00 

$204,432 

$32,485 

$0 

$236,917 

0.00 

W912QR07C001 1 

0.00 

$213,500 

$0 

$0 

$213,500 

0.00 

DACA2701D00 100029 

0.60 

$258,250 

$5,636 

$0 

$263,886 

0.00 

DACA2702C002 1 

0.66 

$3,385,600 

$468,955 

$0 

$3,854,555 

0.00 

DACA2703C00 1 0 

0.49 

$1,342,942 

$59,522 

$0 

$1,402,464 

0.00 

DACW270 1 D00050037 

0.80 

$126,266 

$11,726 

$0 

$137,992 

0.00 

W912QR05C0016 

0.71 

$1,319,925 

$73,346 

$0 

$1,393,271 

0.00 

W912QR05C0020 

0.93 

$1,102,200 

$1,151,426 

$0 

$2,253,626 

0.00 

W912QR05C0030 

0.65 

$614,519 

$18,556 

$0 

$633,075 

0.00 

W9 1 2QR04C00 1 2 

0.00 

$2,650,000 

$506,957 

$0 

$2,697,049 

-0.15 

W912QR05C0025 

0.90 

$212,069 

$0 

$0 

$211,569 

0.00 

W912QR06C0035 

0.70 

$1,685,512 

$25,000 

$0 

$1,694,412 

-0.01 

DACW270 1 D0005000 1 

0.00 

$184,088 

$169,001 

$0 

$353,090 

0.00 

W9 1 2QR06C00 1 4 

0.72 

$5,565,101 

$137,613 

$0 

$5,625,541 

-0.01 

DACW270 1 D000501 26 

0.00 

$121,649 

$4,656 

$0 

$126,306 

0.00 

DACA270 100030059 

0.00 

$180,864 

$7,458 

$0 

$188,322 

0.00 

DAC  A270 1 D00030033 

0.80 

$280,497 

$33,267 

$0 

$313,764 

0.00 

DAC  A270 1 D00030036 

0.11 

$187,000 

$0 

$0 

$187,000 

0.00 

DAC  A270 1 D00030040 

0.83 

$118,735 

$1,300 

$0 

$120,035 

0.00 

DAC  A270 1 D00030047 

0.00 

$288,963 

$232,065 

$0 

$521,028 

0.00 

DAC  A270 1 D00030048 

0.55 

$918,821 

$33,968 

$0 

$927,411 

-0.03 

DAC  A270 1 D0003005 1 

0.00 

$160,277 

$22,507 

$0 

$182,784 

0.00 

DAC  A270 1 D00030054 

0.00 

$915,835 

$109,406 

$0 

$1,002,100 

-0.02 

DAC  A270 1 D00030056 

0.00 

$358,572 

$18,584 

$0 

$375,239 

-0.01 

DAC  A270 1 D00030057 

0.00 

$138,886 

$15,039 

$0 

$153,925 

0.00 

DACA270 1 D00030058 

0.35 

$118,758 

$9,724 

$0 

$128,483 

0.00 

DAC  A270 1 D00030063 

0.00 

$2,660,000 

$119,692 

$0 

$2,779,692 

0.00 

DAC  A270 1 D00030063 

0.00 

$2,660,000 

$119,692 

$0 

$2,779,692 

0.00 

DAC  A270 1 D00030065 

0.00 

$499,538 

$0 

$0 

$499,171 

0.00 

119 


Contract  Number 

%  of  Subcontracting 

Amount  of  Basic 

Contract 

Total  Amount  of 
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DACA2701D00030069 

0.00 

$193,963 

$0 

$0 

$193,963 

0.00 

DACA270 1 D0003007 1 

0.28 

$231,079 

$0 

$0 

$231,079 

0.00 

DACA2701D00030074 

0.00 

$402,980 

$87,141 

$0 

$490,021 

0.00 

DACA270 1 D00030075 

0.00 

$249,820 

$0 

$0 

$249,820 

0.00 

DACA270 1 D00030076 

0.26 

$326,055 

$0 

$0 

$326,055 

0.00 

DACA270 1 D00030077 

0.00 

$328,893 

$0 

$0 

$328,893 

0.00 

DAC  A270 1 D0003007 8 

0.33 

$326,054 

$0 

$0 

$326,054 

0.00 

DACA270 1 D0003008 1 

0.75 

$105,490 

$2,198 

$0 

$107,689 

0.00 

DACA270 1 D0008000 1 

0.80 

$1,204,307 

$0 

$0 

$1,204,307 

0.00 

DAC  A27 99D00040004 

0.00 

$4,089,352 

$297,257 

$0 

$4,386,609 

0.00 

DACW270 1 D00050 1 32 

0.67 

$106,724 

$11,981 

$0 

$118,704 

0.00 

DACW270 1 D00050002 

0.00 

$162,878 

$22,606 

$0 

$185,485 

0.00 

DACW270 1 D00050074 

0.80 

$594,835 

$0 

$0 

$594,835 

0.00 

DACW270 1 D00050 1 02 

0.75 

$266,488 

$0 

$0 

$266,488 

0.00 

DAC  W2702C000 1 

0.00 

$1,589,303 

$278,939 

$0 

$1,851,738 

-0.01 

DACW2702C0015 

0.31 

$12,748,027 

$3,248,915 

$0 

$15,996,842 

0.00 

DACW2703C0008 

0.38 

$215,411,880 

($16,509,562) 

$0 

$194,828,068 

-0.02 

DACW2703C0023 

0.00 

$1,526,984 

$169,918 

$0 

$1,696,902 

0.00 

DACW2703C0026 

0.77 

$5,630,925 

$107,684 

$0 

$5,736,610 

0.00 

W912QR04D00 150005 

0.40 

$301,056 

$62,770 

$0 

$363,726 

0.00 

W912QR04D00 150006 

0.00 

$255,500 

$16,708 

$0 

$272,208 

0.00 

W912QR04D00 150026 

0.95 

$199,401 

$28,998 

$0 

$228,399 

0.00 

W912QR04D00 160004 

0.94 

$102,144 

($1,875) 

$0 

$100,269 

0.00 

W912QR05D00040001 

0.00 

$179,718 

$0 

$0 

$179,718 

0.00 

W912QR05D00070001 

0.00 

$326,052 

$0 

$0 

$326,052 

0.00 

W912QR05D00070002 

0.00 

$31,446 

$0 

$0 

$31,446 

0.00 

W912QR05D00070006 

0.00 

$325,958 

$0 

$0 

$325,958 

0.00 

W912QR05D00070008 

0.00 

$325,959 

$0 

$0 

$325,959 

0.00 

W912QR05D00070009 

0.00 

$325,959 

$0 

$0 

$325,959 

0.00 

W912QR05D0007001 1 

0.00 

$430,794 

$0 

$0 

$430,794 

0.00 

W912QR05D00070012 

0.00 

$104,809 

$0 

$0 

$104,809 

0.00 

W912QR05D00070013 

0.00 

$243,913 

$0 

$0 

$243,913 

0.00 

DACA2703C0020 

0.80 

$4,204,391 

($39,076) 

$0 

$3,675,448 

-0.12 

DACA2703D0008000 1 

0.00 

$193,542 

$0 

$0 

$193,542 

0.00 

DACA2703D00080003 

1.00 

$274,127 

$0 

$0 

$274,127 

0.00 

DACA2703D00080007 

1.00 

$671,818 

$0 

$0 

$671,818 

0.00 

DACA2703D000800 1 8 

0.00 

$145,718 

$0 

$0 

$145,718 

0.00 

DACA2703D000800 1 9 

1.00 

$139,500 

$0 

$0 

$139,500 

0.00 

DACA2703D00080025 

1.00 

$342,908 

$0 

$0 

$342,908 

0.00 

DACA2703D00080026 

0.00 

$549,231 

$13,718 

$0 

$472,949 

-0.16 

DACA2703D00080029 

1.00 

$133,934 

$0 

$0 

$133,934 

0.00 

DACA2703D00080033 

1.00 

$251,083 

$0 

$0 

$251,083 

0.00 

DACA2703D00080039 

1.00 

$223,200 

$0 

$0 

$223,200 

0.00 

DACA2703D00080040 

1.00 

$135,791 

$0 

$0 

$135,791 

0.00 

DACA2703D00080047 

1.00 

$133,312 

$0 

$0 

$133,312 

0.00 

DACA6303C0020 

0.81 

$640,000 

$69,292 

$295 

$319,567 

-0.55 

DACW2701C0022 

0.00 

$424,548 

$155,277 

$0 

$579,825 

0.00 

DACW270 1 D000500 1 5 

0.00 

$135,523 

$0 

$0 

$135,523 

0.00 

DACW270 1 D000500 1 7 

0.00 

$149,549 

$25,801 

$0 

$175,351 

0.00 

DACW270 1 D00050024 

0.98 

$146,722 

$0 

$0 

$146,722 

0.00 

DACW2701D000501 10 

0.77 

$208,917 

$6,945 

$0 

$215,862 

0.00 

DACW2702C0026 

0.00 

$1,183,345 

$15,000 

$0 

$1,198,345 

0.00 

W912QR04C0003 

0.26 

$3,620,000 

$170,836 

$0 

$3,776,296 

0.00 

W9 1 2QR04D0009000 1 

0.77 

$1,307,210 

$121,874 

$0 

$1,429,075 

0.00 

W9 1 2QR04D00 1 500 1 3 

0.00 

$178,696 

$0 

$0 

$176,696 

-0.01 
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W9 1 2QR04D00080003 

0.85 

$1,927,040 

($1,080) 

$0 

$1,925,960 

0.00 

DACA270 1 D0008005 8 

0.09 

$725,966 

$27,037 

$0 

$753,003 

0.00 

W912QR04D00 150007 

0.25 

$280,909 

$34,527 

$0 

$314,436 

0.00 

W912QR04D00 150022 

0.00 

$205,083 

$0 

$0 

$205,083 

0.00 

W912QR04D00 150028 

0.25 

$213,383 

$29,070 

$0 

$242,453 

0.00 

W912QR04D00 160009 

0.00 

$440,615 

$0 

$0 

$440,615 

0.00 

W9 1 2QR05C002 1 

0.80 

$8,207,218 

$141,120 

$0 

$8,321,132 

0.00 

W9 1 2QR06C00 1 7 

0.20 

$3,556,000 

$58,458 

$0 

$3,573,980 

-0.01 

DACA2701D00080056 

0.95 

$139,829 

$0 

$0 

$139,829 

0.00 

DACA2701D00080059 

0.90 

$154,208 

$0 

$0 

$152,634 

-0.01 

DACA2701D00 100008 

0.95 

$81,873 

$242,015 

$0 

$323,888 

0.00 

DACA2703C00 1 8 

0.78 

$8,524,000 

$826,295 

$0 

$9,387,769 

0.00 

W9 1 2QR04D00 1 600 1 7 

0.95 

$38,863 

$0 

$0 

$38,863 

0.00 

W9 1 2QR05C0006 

0.09 

$3,980,000 

$19,095 

$0 

$3,880,467 

-0.03 

W912QR04D00 160033 

0.00 

$124,000 

$0 

$0 

$124,000 

0.00 

W9 1 2QR04D0025000 1 

0.63 

$1,075,150 

$823,655 

$0 

$1,898,805 

0.00 

W912QR05C0034 

0.01 

$764,112 

$5,875 

$0 

$769,987 

0.00 

W9 1 2QR05D00040002 

0.00 

$175,014 

$0 

$0 

$175,014 

0.00 

W9 1 2QR05D00040003 

0.00 

$780,001 

$0 

$0 

$769,253 

-0.01 

W9 1 2QR05D000700 1 7 

0.00 

$255,000 

$79 

$0 

$254,920 

0.00 

W9 1 2QR05D00070020 

0.28 

$1,451,043 

$0 

$0 

$1,052,640 

-0.27 

W9 1 2QR05D00070022 

0.45 

$183,161 

$0 

$0 

$183,161 

0.00 

W9 1 2QR05D00070024 

0.62 

$283,000 

$0 

$0 

$267,813 

-0.05 

W9 1 2QR05D00070025 

0.47 

$1,048,000 

$0 

$0 

$0 

-1.00 

W9 1 2QR06C0020 

0.45 

$5,027,500 

$693,363 

$0 

$5,720,863 

0.00 

W9 1 2QR06C0057 

0.30 

$241,857 

$4,618 

$0 

$235,640 

-0.04 

DACW2701C0030 

0.04 

$24,156,000 

$470,785 

$0 

$24,626,785 

0.00 

DACW270 1 D000500 1 6 

0.30 

$749,831 

$215,000 

$0 

$964,831 

0.00 

DACW270 1 D00050087 

0.00 

$111,952 

$0 

$0 

$111,952 

0.00 

DACW270 1 D00050092 

1.00 

$1,537,641 

($173,850) 

$0 

$1,361,841 

0.00 

DACW270 1 D00050093 

0.99 

$919,849 

$321,578 

$0 

$1,241,426 

0.00 

DACW270 1 D00050094 

0.99 

$113,825 

$0 

$0 

$113,825 

0.00 

DACW270 1 D00050095 

1.00 

$629,465 

$0 

$0 

$629,465 

0.00 

DACW2702C0005 

0.30 

$8,696,883 

$3,153,117 

$0 

$11,786,029 

-0.01 

W912QR04D00 150003 

0.60 

$156,710 

$24,185 

$0 

$180,895 

0.00 

W912QR04D00 160030 

1.00 

$308,900 

$0 

$0 

$308,900 

0.00 

W912QR06D0008001 1 

0.80 

$1,265,000 

($118,466) 

$0 

$1,146,434 

0.00 

W9 1 2QR06D00080036 

0.10 

$1,233,375 

($10,001) 

$0 

$1,223,373 

0.00 

W9 1 2QR07C004 1 

1.00 

$179,400 

$0 

$0 

$174,477 

-0.03 

W9 1 2QR06C00 1 1 

0.98 

$2,269,884 

$23,654 

$0 

$2,222,741 

-0.03 

W9 1 2QR06C00 1 6 

0.95 

$1,409,850 

$2,504 

$0 

$1,330,452 

-0.06 

DAC  W2702C00 1 8 

0.00 

$3,064,200 

$452,365 

$0 

$3,497,850 

-0.01 

W9 1 2D06D000 1 C  Y0 1 

0.04 

$964,502 

$184,311 

$0 

$1,148,813 

0.00 

W912QR04D00 150030 

0.05 

$1,542,840 

$28,352 

$0 

$1,571,192 

0.00 

W9 1 2QR04C0024 

0.00 

$10,109,000 

$485,923 

$0 

$10,565,423 

0.00 

W912QR04D00 140006 

0.70 

$1,520,000 

$509,138 

$0 

$1,868,606 

-0.08 

W912QR04D00 150031 

0.67 

$1,837,191 

$6,546 

$0 

$1,843,737 

0.00 

W912QR04D00 150033 

0.80 

$330,000 

($22,343) 

$0 

$307,558 

0.00 

W9 1 2QR04D00 1 600 1 6 

0.00 

$952,800 

$215,502 

$0 

$1,168,302 

0.00 

W9 1 2QR06C0029 

0.37 

$2,620,000 

$21,221 

$0 

$2,641,221 

0.00 

DACA2703C0007 

0.71 

$83,329,000 

$3,039,877 

$0 

$86,368,877 

0.00 

W9 1 2QR06C0039 

0.47 

$5,855,387 

$582,721 

$0 

$6,218,816 

-0.03 

W9 1 2QR06C0040 

0.87 

$3,095,554 

$105,329 

$0 

$3,200,883 

0.00 

W9 1 2QR06D000800 1 7 

0.00 

$172,530 

$48,764 

$0 

$221,294 

0.00 

W9 1 2QR06D000800 1 8 

0.00 

$81,900 

$36,349 

$0 

$118,249 

0.00 
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DACA2700D00050019 

3/15/2001 

6/1/2002 

5/31/2006 

1903 

6/27/2006 

1930 

DACA2700D00050157 

4/4/2002 

9/26/2002 

11/10/2002 

220 

9/16/2003 

530 

DACA270 1 D00080003 

9/7/2001 

8/21/2002 

10/11/2002 

399 

10/8/2002 

396 

DAC  A27 0 1 D00080040 

6/20/2003 

12/17/2003 

4/15/2004 

300 

4/15/2004 

300 

DACA2702C00 1 3 

5/31/2002 

6/24/2004 

9/7/2004 

830 

9/7/2004 

830 

DACA2703D0008007 1 

8/17/2005 

6/25/2006 

6/25/2006 

312 

12/31/2005 

136 

DACA2703D00080072 

9/22/2005 

7/27/2006 

7/27/2006 

308 

7/27/2006 

308 

DACA2703D00080079 

9/30/2005 

10/17/2006 

10/17/2006 

382 

5/31/2006 

243 

DACW2700C00 1 6 

4/13/2000 

11/22/2001 

11/22/2001 

588 

8/11/2004 

1581 

DACW2703C0025 

9/30/2003 

1 1/1 1/2004 

5/25/2006 

968 

10/3/2006 

1099 

W9 1 2QR04D00 1 600 1 1 

1/6/2005 

4/30/2005 

4/30/2005 

114 

4/29/2005 

113 

W912QR04D00 160008 

9/24/2004 

12/10/2005 

12/31/2005 

463 

1/11/2007 

839 

W912QR06C0010 

2/15/2006 

9/4/2007 

9/4/2007 

566 

10/30/2007 

622 

W912QR06C0044 

8/30/2006 

2/5/2008 

3/17/2008 

565 

3/18/2008 

566 

W9 1 2QR07  C0040 

6/26/2007 

6/20/2008 

6/20/2008 

360 

6/12/2008 

352 

D  AC  A27 01D00090009 

12/22/2003 

2/21/2006 

4/30/2006 

860 

6/5/2006 

896 

D  AC  W270 1 D00050029 

9/23/2002 

2/18/2003 

6/30/2003 

280 

12/1/2003 

434 

W912QR04D00150014 

2/17/2005 

7/13/2005 

7/15/2006 

513 

3/19/2007 

760 

W9 1 2QR04D00 150014 

2/18/2005 

3/15/2006 

3/8/2006 

383 

3/8/2006 

383 

W9 1 2QR05C00 1 3 

5/27/2005 

5/16/2006 

9/22/2006 

483 

10/3/2006 

494 

W912QR06D00080040 

7/17/2007 

9/2/2007 

9/2/2007 

47 

8/30/2007 

44 

W912QR06C0008 

2/16/2006 

10/7/2007 

1 1/28/2007 

650 

1 1/28/2007 

650 

DACW270 1 D00050052 

1/31/2003 

6/14/2003 

6/14/2003 

134 

10/9/2003 

251 

DACW270 1 D00050078 

6/20/2003 

11/13/2003 

11/13/2003 

146 

3/16/2005 

635 

W912QR04D00 160027 

9/29/2006 

8/27/2007 

8/27/2007 

332 

8/10/2007 

315 

W912QR06D00080007 

7/21/2006 

9/30/2006 

6/30/2007 

344 

6/26/2007 

340 

W912QR06D00080032 

3/20/2007 

8/8/2007 

8/29/2007 

162 

9/4/2007 

168 

W912QR07C001 1 

6/7/2007 

8/31/2007 

8/31/2007 

85 

10/4/2007 

119 

DACA2701D00 100029 

9/30/2005 

2/21/2006 

2/21/2006 

144 

1/25/2006 

117 

DACA2702C002 1 

8/28/2002 

12/17/2003 

6/30/2004 

672 

6/30/2004 

672 

DACA2703C00 1 0 

6/23/2003 

5/12/2004 

1 1/23/2004 

519 

1 1/23/2004 

519 

D  AC  W270 1 D00050037 

9/28/2002 

12/17/2002 

12/17/2002 

80 

12/11/2002 

74 

W9 1 2QR05C00 1 6 

6/17/2005 

6/30/2006 

7/22/2006 

400 

7/18/2006 

396 

W9 1 2QR05C0020 

6/30/2005 

4/8/2006 

4/13/2006 

287 

4/13/2006 

287 

W9 1 2QR05C0030 

9/14/2005 

5/31/2006 

9/1/2006 

352 

5/2/2006 

230 

W9 1 2QR04C00 1 2 

5/21/2004 

5/2/2005 

11/18/2005 

546 

1 1/30/2005 

558 

W9 1 2QR05C0025 

8/16/2005 

5/21/2006 

5/21/2006 

278 

5/18/2006 

275 

W9 1 2QR06C0035 

9/29/2006 

9/25/2007 

10/30/2007 

396 

10/29/2007 

395 

DACW270 1 D0005000 1 

9/30/2001 

1 1/26/2002 

12/26/2002 

452 

12/26/2002 

452 

W9 1 2QR06C00 1 4 

2/27/2006 

7/13/2007 

9/17/2007 

567 

9/17/2007 

567 

DACW270 1 D00050 1 26 

5/4/2004 

10/16/2004 

10/16/2004 

165 

10/1/2004 

150 

DACA270 100030059 

6/18/2003 

2/13/2004 

7/12/2004 

390 

7/16/2004 

394 

DACA270 1 D00030033 

6/21/2002 

1/17/2003 

7/25/2005 

1130 

7/13/2005 

1118 

DACA270 1 D00030036 

8/23/2002 

1/5/2003 

1/4/2005 

865 

12/2/2004 

832 

DACA270 1 D00030040 

8/30/2002 

2/6/2003 

3/8/2003 

190 

1 1/10/2003 

437 

DACA270 1 D00030047 

11/1/2002 

4/30/2003 

5/31/2004 

577 

12/2/2004 

762 

DACA270 1 D00030048 

12/30/2002 

7/6/2004 

2/1/2005 

764 

1/27/2005 

759 

DACA270 1 D0003005 1 

2/11/2003 

6/1 1/2003 

3/31/2004 

414 

4/19/2006 

1163 

DACA270 1 D00030054 

4/1/2003 

6/30/2003 

8/14/2003 

135 

12/1/2003 

244 

DACA270 1 D00030056 

4/18/2003 

9/15/2003 

12/14/2003 

240 

3/1/2004 

318 

DACA270 1 D00030057 

5/15/2003 

8/13/2003 

1/10/2004 

240 

8/23/2004 

466 

DAC  A270 1 D0003005 8 

6/4/2003 

11/13/2003 

12/13/2003 

192 

6/22/2004 

384 

DAC  A27 0 1 D00030063 

9/3/2003 

12/28/2004 

2/6/2006 

887 

6/22/2006 

1023 

DAC  A27 0 1 D00030063 

9/3/2003 

12/28/2004 

2/6/2006 

887 

6/22/2006 

1023 

DAC  A27 0 1 D00030065 

9/16/2003 

1/14/2004 

1/14/2004 

120 

7/16/2004 

304 
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DACA270 1 D00030069 

9/30/2003 

3/28/2004 

12/13/2004 

440 

2/13/2006 

867 

DACA270 1 D0003007 1 

4/1/2004 

9/28/2004 

9/28/2004 

180 

5/16/2005 

410 

DACA270 1 D00030074 

9/9/2004 

3/8/2005 

3/8/2005 

180 

5/9/2005 

242 

DACA270 1 D00030075 

9/9/2004 

3/8/2005 

3/8/2005 

180 

2/7/2006 

516 

DACA270 1 D00030076 

10/6/2004 

1/4/2005 

1/4/2005 

90 

12/10/2004 

65 

DAC  A27 0 1 D00030077 

10/29/2004 

7/26/2005 

7/26/2005 

270 

2/13/2006 

472 

DAC  A270 1 D0003007 8 

12/11/2004 

3/11/2005 

3/11/2005 

90 

3/11/2005 

90 

DACA270 1 D0003008 1 

3/17/2005 

1 1/12/2005 

1 1/12/2005 

240 

9/30/2005 

197 

DACA270 1 D0008000 1 

9/7/2001 

8/15/2002 

5/12/2003 

612 

7/28/2003 

689 

D  AC  A27 99D00040004 

12/23/2002 

8/22/2004 

2/5/2005 

775 

5/16/2005 

875 

DACW270 1 D00050 1 32 

7/8/2004 

10/17/2004 

10/17/2004 

101 

6/30/2005 

357 

DACW270 1 D00050002 

10/24/2001 

12/8/2001 

12/22/2001 

59 

1 1/29/2001 

36 

D  AC  W270 1 D00050074 

6/23/2003 

1 1/6/2003 

11/6/2003 

136 

8/1/2005 

770 

DACW270 1 D00050 1 02 

11/4/2003 

5/22/2004 

5/22/2004 

200 

6/30/2005 

604 

DACW2702C000 1 

11/2/2001 

12/13/2002 

1/17/2003 

441 

3/31/2004 

880 

DACW2702C00 1 5 

5/30/2002 

6/27/2004 

1/5/2006 

1316 

12/8/2005 

1288 

DACW2703C0008 

2/26/2003 

3/27/2004 

3/4/2005 

737 

6/25/2004 

485 

DACW2703C0023 

9/16/2003 

10/2/2004 

10/2/2004 

382 

12/10/2004 

451 

DACW2703C0026 

9/29/2003 

1/1/2005 

3/5/2005 

523 

3/5/2005 

523 

W912QR04D00 150005 

7/2/2004 

9/30/2004 

9/30/2006 

820 

9/29/2006 

819 

W912QR04D00 150006 

7/6/2004 

10/16/2004 

10/16/2004 

102 

12/11/2006 

888 

W912QR04D00 150026 

6/10/2005 

9/30/2005 

9/30/2005 

112 

8/18/2006 

434 

W912QR04D00 160004 

7/29/2004 

9/30/2005 

6/30/2005 

336 

8/19/2005 

386 

W9 1 2QR05D0004000 1 

3/31/2005 

10/16/2005 

10/16/2005 

199 

10/16/2005 

199 

W9 1 2QR05D0007000 1 

6/17/2005 

9/18/2005 

9/18/2005 

93 

3/24/2006 

280 

W912QR05D00070002 

7/18/2005 

12/15/2005 

12/15/2005 

150 

3/13/2006 

238 

W9 1 2QR05D00070006 

9/15/2005 

12/14/2005 

12/14/2005 

90 

12/14/2005 

90 

W912QR05D00070008 

12/8/2005 

3/9/2006 

3/9/2006 

91 

1/4/2007 

392 

W9 1 2QR05D00070009 

3/6/2006 

6/8/2006 

6/8/2006 

94 

1/3/2007 

303 

W912QR05D0007001 1 

6/2/2006 

9/30/2006 

9/30/2006 

120 

4/10/2007 

312 

W9 1 2QR05D000700 1 2 

6/14/2006 

9/12/2006 

1/30/2007 

230 

4/19/2007 

309 

W912QR05D00070013 

8/2/2006 

10/1/2006 

10/1/2006 

60 

1/17/2007 

168 

DACA2703C0020 

9/24/2003 

10/13/2004 

1 1/26/2004 

429 

12/14/2004 

447 

DACA2703D0008000 1 

9/26/2003 

12/28/2003 

12/28/2003 

93 

12/26/2003 

91 

DACA2703D00080003 

9/27/2003 

3/2/2004 

4/1/2004 

187 

3/31/2004 

186 

DACA2703D00080007 

9/29/2003 

4/12/2004 

2/28/2005 

518 

3/17/2005 

535 

DACA2703D000800 1 8 

3/5/2004 

8/6/2004 

8/6/2004 

154 

7/31/2004 

148 

DACA2703D000800 1 9 

3/19/2004 

8/6/2004 

2/28/2005 

346 

3/10/2005 

356 

DACA2703D00080025 

6/1/2004 

2/19/2005 

2/19/2005 

263 

10/27/2004 

148 

DACA2703D00080026 

6/3/2004 

12/18/2004 

12/18/2004 

198 

1 1/30/2004 

180 

DACA2703D00080029 

8/13/2004 

12/21/2004 

12/21/2004 

130 

10/31/2004 

79 

DACA2703D00080033 

8/20/2004 

12/28/2004 

12/28/2004 

130 

1 1/29/2004 

101 

DACA2703D00080039 

9/8/2004 

3/14/2005 

3/15/2005 

188 

3/14/2005 

187 

DACA2703D00080040 

9/9/2004 

1/18/2005 

3/19/2005 

191 

6/1/2005 

265 

DACA2703D00080047 

9/20/2004 

1/28/2005 

3/31/2005 

192 

3/31/2005 

192 

DACA6303C0020 

9/20/2003 

2/24/2004 

7/21/2004 

305 

6/1/2005 

620 

DACW2701C0022 

6/22/2001 

1/16/2002 

6/22/2002 

365 

11/1/2001 

132 

DACW270 1 D000500 1 5 

4/17/2002 

8/15/2002 

11/8/2002 

205 

12/3/2002 

230 

DACW270 1 D000500 1 7 

5/15/2002 

9/12/2002 

10/30/2002 

168 

10/30/2002 

168 

D  AC  W270 1 D00050024 

8/8/2002 

9/30/2002 

9/30/2002 

53 

9/30/2002 

53 

DACW2701D000501 10 

1/15/2004 

8/1/2004 

8/1/2004 

199 

8/1/2004 

199 

DACW2702C0026 

9/30/2002 

7/21/2003 

10/13/2003 

378 

7/20/2006 

1389 

W9 1 2QR04C0003 

12/18/2003 

1/13/2006 

1/13/2006 

757 

3/24/2005 

462 

W9 1 2QR04D0009000 1 

8/12/2004 

12/31/2004 

8/1/2005 

354 

7/26/2005 

348 

W9 1 2QR04D00 150013 

2/1/2005 

4/30/2005 

4/30/2005 

88 

9/21/2005 

232 

123 


Contract  Number 

Date  of  Award 

Original  Contract 
Completion  Date 

Revised  Contract 
Completion  Date 

Scheduled  Days 
Work 

Date  Work 
Accepted 

Actual  Days 
Work 

W912QR04D00080003 

9/26/2005 

8/25/2006 

9/19/2006 

358 

9/8/2006 

347 

DAC  A270 1 D0008005 8 

9/28/2004 

10/21/2005 

10/21/2005 

388 

10/17/2005 

384 

W912QR04D00 150007 

7/9/2004 

10/16/2004 

10/16/2004 

99 

10/16/2004 

99 

W912QR04D00 150022 

3/31/2005 

8/11/2005 

8/11/2005 

133 

8/4/2005 

126 

W912QR04D00 150028 

6/13/2005 

9/30/2005 

9/30/2005 

109 

9/30/2005 

109 

W912QR04D00 160009 

12/10/2004 

12/29/2004 

12/29/2004 

19 

1 1/19/2004 

-21 

W9 1 2QR05C002 1 

6/30/2005 

1/29/2007 

1/29/2007 

578 

1/18/2007 

567 

W9 1 2QR06C00 1 7 

3/28/2006 

10/18/2006 

11/7/2006 

224 

11/7/2006 

224 

DACA270 1 D00080056 

9/3/2004 

12/31/2004 

12/31/2004 

119 

12/2/2004 

90 

DACA270 1 D00080059 

9/30/2004 

1/19/2005 

5/19/2005 

231 

5/20/2005 

232 

DACA2701D00 100008 

8/15/2003 

7/18/2004 

2/23/2005 

558 

9/22/2005 

769 

DACA2703C00 1 8 

9/17/2003 

7/5/2005 

7/5/2005 

657 

6/30/2005 

652 

W9 1 2QR04D00 1 600 1 7 

6/2/2005 

9/14/2005 

9/14/2005 

104 

10/14/2005 

134 

W9 1 2QR05C0006 

3/11/2005 

9/13/2005 

9/22/2005 

195 

9/22/2005 

195 

W912QR04D00 160033 

9/4/2007 

12/5/2007 

12/5/2007 

92 

4/15/2008 

224 

W9 1 2QR04D0025000 1 

9/28/2005 

9/30/2007 

3/1/2008 

885 

2/28/2008 

883 

W9 1 2QR05C0034 

9/16/2005 

6/18/2006 

6/18/2006 

275 

6/15/2006 

272 

W9 1 2QR05D00040002 

11/4/2005 

7/11/2006 

7/11/2006 

249 

1/8/2008 

795 

W9 1 2QR05D00040003 

2/28/2007 

12/1/2007 

12/1/2007 

276 

5/8/2008 

435 

W9 1 2QR05D000700 1 7 

11/9/2006 

9/5/2007 

9/5/2007 

300 

9/5/2007 

300 

W9 1 2QR05D00070020 

9/13/2007 

5/23/2008 

5/23/2008 

253 

5/9/2008 

239 

W9 1 2QR05D00070022 

9/19/2007 

12/23/2007 

12/23/2007 

95 

11/2/2007 

44 

W9 1 2QR05D00070024 

9/28/2007 

8/10/2008 

8/10/2008 

317 

1/25/2008 

119 

W9 1 2QR05D00070025 

9/28/2007 

9/24/2008 

9/24/2008 

362 

4/28/2008 

213 

W9 1 2QR06C0020 

4/21/2006 

1 1/3/2007 

2/28/2008 

678 

2/22/2008 

672 

W9 1 2QR06C0057 

9/30/2006 

2/21/2007 

5/2/2007 

214 

3/7/2007 

158 

DACW2701C0030 

9/27/2001 

7/7/2005 

7/7/2005 

1379 

7/7/2005 

1379 

DACW270 1 D000500 1 6 

5/13/2002 

7/5/2002 

7/5/2002 

53 

7/5/2002 

53 

DACW270 1 D00050087 

8/7/2003 

10/14/2003 

10/14/2003 

68 

4/9/2004 

246 

D  AC  W270 1 D00050092 

9/4/2003 

9/24/2004 

10/21/2004 

413 

8/31/2005 

727 

DACW270 1 D00050093 

9/8/2003 

1/24/2004 

7/30/2004 

326 

12/10/2004 

459 

D  AC  W270 1 D00050094 

9/16/2003 

1 1/10/2003 

1 1/20/2003 

65 

10/22/2003 

36 

D  AC  W270 1 D00050095 

9/30/2003 

4/1 1/2004 

4/11/2004 

194 

1/31/2005 

489 

DACW2702C0005 

3/8/2002 

7/1/2004 

10/28/2005 

1330 

10/28/2005 

1330 

W912QR04D00 150003 

6/18/2004 

12/5/2004 

1/4/2005 

200 

6/20/2007 

1097 

W912QR04D00 160030 

7/17/2007 

11/12/2007 

1 1/12/2007 

118 

10/18/2007 

93 

W9 1 2QR06D000800 1 1 

8/3/2006 

11/12/2006 

1/1/2007 

151 

12/31/2006 

150 

W9 1 2QR06D00080036 

6/1/2007 

11/19/2007 

1 1/19/2007 

171 

8/27/2007 

87 

W912QR07C0041 

6/28/2007 

1/12/2008 

1/12/2008 

198 

10/19/2007 

113 

W9 1 2QR06C00 1 1 

2/16/2006 

2/13/2007 

2/13/2007 

362 

2/9/2007 

358 

W9 1 2QR06C00 1 6 

3/2/2006 

3/1/2007 

3/1/2007 

364 

2/9/2007 

344 

DACW2702C00 1 8 

8/16/2002 

10/16/2003 

1 1/29/2004 

836 

10/29/2004 

805 

W9 1 2D06D000 1 CYO 1 

6/30/2006 

11/15/2006 

11/15/2006 

138 

8/22/2007 

418 

W912QR04D00 150030 

6/27/2005 

5/10/2006 

6/2/2006 

340 

4/26/2006 

303 

W9 1 2QR04C0024 

9/9/2004 

3/17/2006 

9/6/2006 

727 

10/27/2006 

778 

W912QR04D00 140006 

9/30/2005 

9/20/2006 

7/9/2007 

647 

6/21/2007 

629 

W912QR04D00 150031 

6/27/2005 

5/10/2006 

7/13/2006 

381 

7/11/2006 

379 

W912QR04D00 150033 

4/6/2006 

8/30/2006 

8/30/2006 

146 

8/9/2006 

125 

W9 1 2QR04D00 1 600 1 6 

5/24/2005 

12/10/2005 

5/3/2006 

344 

5/23/2006 

364 

W9 1 2QR06C0029 

5/26/2006 

2/16/2007 

6/5/2007 

375 

6/5/2007 

375 

DACA2703C0007 

5/21/2003 

11/27/2006 

4/7/2007 

1417 

4/6/2007 

1416 

W9 1 2QR06C0039 

8/24/2006 

4/10/2008 

6/30/2008 

676 

6/28/2008 

674 

W9 1 2QR06C0040 

8/16/2006 

8/14/2007 

9/25/2007 

405 

9/24/2007 

404 

W9 1 2QR06D000800 1 7 

9/15/2006 

10/31/2006 

9/30/2007 

380 

8/27/2008 

712 

W9 1 2QR06D000800 1 8 

9/18/2006 

10/31/2006 

12/15/2006 

88 

12/15/2006 

88 

124 


Contract  Number 

Project  Schedule  Growth 

Overall 

Rating 

DACA2700D000500 1 9 

0.01 

4 

DACA2700D00050 1 57 

1.41 

DACA270 1 D00080003 

-0.01 

DACA270 1 D00080040 

0.00 

3 

D  AC  A2702C00 1 3 

0.00 

3 

DACA2703D0008007 1 

-0.56 

4 

D  AC  A27 03 D0008007 2 

0.00 

3 

D  AC  A27 03 D0008007 9 

-0.36 

4 

D  AC  W 27 00C00 1 6 

1.69 

5 

DACW2703C0025 

0.14 

W912QR04D00 16001 1 

-0.01 

W912QR04D00 160008 

0.81 

W912QR06C0010 

0.10 

W9 1 2QR06C0044 

0.00 

W9 1 2QR07C0040 

-0.02 

4 

DACA270 1 D00090009 

0.04 

3 

DACW2701D00050029 

0.55 

3 

W912QR04D00150014 

0.48 

4 

W912QR04D00150014 

0.00 

4 

W912QR05C0013 


W9 1 2QR06D00080040 

-0.06 

W9 1 2QR06C0008 

0.00 

D  AC  W27 0 1 D00050052 

0.87 

4 

DACW2701D00050078 

3.35 

3 

W912QR04D00 160027 

-0.05 

3 

W9 1 2QR06D00080007 

-0.01 

4 

W912QR06D00080032 

0.04 

4 

W912QR07C001 1 

0.40 

4 

DACA2701D00 100029 

-0.19 

5 

D  AC  A2702C002 1 

0.00 

2 

D  AC  A2703C00 1 0 

0.00 

5 

D  AC  W27 0 1 D00050037 

-0.08 

4 

W912QR05C0016 

-0.01 

5 

W9 1 2QR05C0020 

0.00 

W912QR05C0030 

-0.35 

W912QR04C0012 

0.02 

4 

W9 1 2QR05C0025 

-0.01 

3 

W912QR06C0035 

0.00 

4 

D  AC  W270 1 D0005000 1 

0.00 

3 

W912QR06C0014 

0.00 

4 

D  AC  W270 1 D00050 1 26 

-0.09 

3 

DACA270 100030059 

0.01 

DACA270 1 D00030033 

-0.01 

DACA270 1 D00030036 

-0.04 

DAC  A270 1 D00030040 

1.30 

4 

DACA270 1 D00030047 

0.32 

4 

DACA270 1 D00030048 

-0.01 

3 

DACA270 1 D0003005 1 

1.81 

3 

DACA270 1 D00030054 

0.81 

4 

DACA270 1 D00030056 

0.33 

DAC  A270 1 D00030057 

0.94 

DACA270 1 D00030058 

1.00 

DACA270 1 D00030063 

0.15 

3 

DAC  A270 1 D00030063 

0.15 

4 

Contract  Number 

DACA270 1 D00030069 


Overall 

Project  Schedule  Growth 

Rating 

15. a 

15.b 

15.c 

15.d 

15.e 

15.f 

15.g 

15.h 

15.1 

15.j 

15.k 

DACA270 1 D00030069 

0.97 

3 

DACA270 1 D0003007 1 

1.28 

3 

DACA270 1 D00030074 

0.34 

3 

DACA270 1 D00030075 

1.87 

3 

DACA270 1 D00030076 

-0.28 

4 

DACA270 1 D00030077 

0.75 

3 

DACA270 1 D00030078 

0.00 

3 

DACA270 1 D0003008 1 

-0.18 

3 

DACA270 1 D0008000 1 

0.13 

3 

DACA2799D00040004 

0.13 

4 

D  AC  W27 0 1 D00050 1 32 

2.53 

3 

DACW2701D00050002 

-0.39 

3 

D  AC  W270 1 D00050074 

4.66 

4 

D  AC  W27 0 1 D00050 1 02 

2.02 

3 

DACW2702C0001 

1.00 

3 

DACW2702C0015 

-0.02 

3 

DACW2703C0008 

-0.34 

4 

DACW2703C0023 

0.18 

3 

DACW2703C0026 

0.00 

4 

W912QR04D00 150005 

0.00 

4 

W912QR04D00 150006 

7.71 

3 

W912QR04D00 150026 

2.88 

3 

W912QR04D00 160004 

0.15 

3 

W9 1 2QR05D0004000 1 

0.00 

3 

W9 1 2QR05D0007000 1 

2.01 

3 

W912QR05D00070002 

0.59 

4 

W912QR05D00070006 

0.00 

3 

W912QR05D00070008 

3.31 

3 

W912QR05D00070009 

2.22 

3 

W9 1 2QR05D000700 1 1 

1.60 

3 

W912QR05D00070012 

0.34 

3 

W9 1 2QR05D000700 1 3 

1.80 

3 

DACA2703C0020 

0.04 

4 

DACA2703D0008000 1 

-0.02 

3 

D  AC  A27 03 D00080003 

-0.01 

4 

D  AC  A27 03 D00080007 

0.03 

3 

DACA2703D000800 1 8 

-0.04 

4 

D  AC  A27 03 D000800 1 9 

0.03 

4 

DACA2703D00080025 

-0.44 

3 

D  AC  A27 03 D00080026 

-0.09 

3 

D  AC  A27 03 D00080029 

-0.39 

3 

D  AC  A27 03 D0008003 3 

-0.22 

3 

DACA2703D00080039 

-0.01 

4 

D  AC  A27 03 D00080040 

0.39 

3 

D  AC  A27 03 D00080047 

0.00 

4 

DACA6303C0020 

1.03 

2 

D  AC  W 2701 C0022 

-0.64 

3 

D  AC  W27 0 1 D000500 1 5 

0.12 

3 

D  AC  W27 0 1 D000500 1 7 

0.00 

3 

DACW2701D00050024 

0.00 

3 

DACW2701D000501 10 

0.00 

3 

DACW2702C0026 

2.67 

3 

W9 1 2QR04C0003 


-0.39 


W9 1 2QR04D0009000 1 


W9 1 2QR04D00 150013 


3 


W912QR04D00 150022 
W912QR04D00 150028 
W912QR04D00 160009 


W912QR05C0021 
W912QR06C0017 
DACA270 1 D00080056 


DACA270 1 D00080059 
DACA2701D00 100008 


W9 1 2QR04D00 1 6003 3 
W9 1 2QR04D0025000 1 
W912QR05C0034 


W9 1 2QR05  D000700 1 7 


W9 1 2QR05D00070020 
W9 1 2QR05D00070022 


W9 1 2QR06C0057 
DACW2701C0030 
D  AC  W27 0 1 D000500 1 6 
D  AC  W27 0 1 D00050087 
DACW2701D00050092 
DACW2701D00050093 
D  AC  W27 0 1 D00050094 
DACW2701D00050095 
DACW2702C0005 
W912QR04D00 150003 
W912QR04D00 160030 
W9 1 2QR06D000800 1 1 
W9 1 2QR06D00080036 
W912QR07C0041 
W912QR06C001 1 
W9 1 2QR06C00 1 6 


W912QR04D00 150030 


W9 1 2QR06D000800 1 8 


15.b  15.c  15.d  15.e 


15.f  |  15.g 


0 

D  AC  A2703C00 1 8 

-0.01 

W9 1 2QR04D00 1 600 1 7 

0.29 

W9 1 2QR05C0006 

0.00 

0 

W9 1 2QR05D00040002 

2.19 

W9 1 2QR05D00040003 

0.58 

W9 1 2QR05D00070024 

-0.62 

W912QR05D00070025 

-0.41 

W9 1 2QR06C0020 

-0.01 

DACW2702C0018 

-0.04 

W9 1 2D06D000 1 C  Y  0 1 

2.03 

0 

W9 1 2QR04C0024 

0.07 

5 

W9 1 2QR04D00 1 40006 

-0.03 

4 

W912QR04D00 150031 

-0.01 

W912QR04D00 150033 

-0.14 

W9 1 2QR04D00 1 600 1 6 

0.06 

W9 1 2QR06C0029 

0.00 

DACA2703C0007 

0.00 

W9 1 2QR06C0039 

0.00 

W9 1 2QR06C0040 

0.00 

4 

W9 1 2QR06D000800 1 7 

0.87 

3 

DACW2700C00 1 6 

5 

4 

3 

DACW2703C0025 

3 

3 

3 

W912QR04D00 16001 1 

3 

3 

3 

W912QR04D00 160008 

3 

3 

3 

W912QR06C0010 

4 

3 

3 

W9 1 2QR06C0044 

4 

3 

4 

W9 1 2QR07C0040 

4 

4 

3 

DAC  A270 1 D00090009 

3 

2 

2 

D  AC  W270 1 D00050029 

3 

3 

3 

W9 1 2QR04D00 150014 

5 

4 

5 

W9 1 2QR04D00 1 50014 


W912QR05C0013 


W9 1 2QR06D00080040 

4 

3 

4 

W9 1 2QR06C0008 

3 

3 

3 

D  AC  W270 1 D00050052 

5 

4 

4 

D  AC  W270 1 D0005007 8 

3 

3 

3 

W912QR04D00 160027 

3 

3 

3 

W9 1 2QR06D00080007 

5 

5 

3 

W912QR06D00080032 

5 

3 

0 

W9 1 2QR07C00 1 1 

5 

4 

0 

DACA2701D00 100029 

5 

4 

5 

DACA2702C002 1 


DACA2703C00 1 0 


D  AC  W270 1 D00050037 

4 

3 

3 

W912QR05C0016 

5 

5 

5 

W912QR05C0020 

4 

3 

3 

W9 1 2QR05C0030 

5 

4 

4 

W912QR04C0012 

5 

4 

4 

W9 1 2QR05C0025 

3 

3 

2 

W9 1 2QR06C0035 

5 

4 

3 

D  AC  W270 1 D0005000 1 

3 

3 

3 

W912QR06C0014 

5 

4 

3 

DACW270 1 D00050 1 26 


DACA270 100030059 


DAC  A270 1 D00030033 


DAC  A270 1 D00030036 

3 

3 

3 

DAC  A270 1 D00030040 

4 

3 

3 

DACA2701D00030047 

5 

5 

4 

DAC  A270 1 D00030048 

3 

3 

3 

DAC  A270 1 D0003005 1 

3 

2 

2 

DAC  A270 1 D00030054 

5 

5 

4 

DACA2701D00030056 

3 

3 

3 

DACA2701D00030057 

3 

3 

2 

DAC  A270 1 D00030058 

3 

3 

3 

DAC  A270 1 D00030063 


DACA2701D00030063 


DAC  A270 1 D00030065 


128 


DAC  A2703D0008000 1 

3 

DACA2703D00080003 

4 

DACA2703D00080007 

3 

DAC  A2703D000800 1 8 

4 

DAC  A2703D000800 1 9 

4 

DACA2703D00080025 

3 

DACA2703D00080026 

4 

DACA2703D00080029 

4 

DACA2703D00080033 

4 

DACA2703D00080039 

4 

DACA2703D00080040 

4 

DACA2703D00080047 

4 

DACA6303C0020 

2 

DACW270 1 C0022 

3 

DACW270 1 D000500 1 5 

3 

DACW270 1 D000500 1 7 


DACW270 1 D00050024 


DACW270 1 D00050 110 


DACW2702C0026 


W9 1 2QR04C0003 
W912QR04D00090001 
W9 1 2QR04D00 150013 
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Contract  Number 

16. a 

16.b 

16.c 

16.d 

16.e 

16.f 

16.g 

16.h 

16.i 

■ 

17.b 

■ 

17.d 

17.e 

17.f 

17.g 

W9 1 2QR04D00080003 

3 

4 

4 

3 

4 

3 

3 

4 

3 

4 

4 

5 

4 

0 

4 

0 

DACA270 1 D00080058 

4 

4 

4 

4 

4 

4 

4 

4 

0 

3 

3 

3 

3 

4 

3 

0 

W912QR04D00 150007 

5 

5 

5 

4 

4 

3 

5 

5 

4 

4 

3 

5 

3 

4 

3 

4 

W912QR04D00 150022 

5 

4 

5 

4 

4 

3 

5 

5 

4 

3 

3 

4 

3 

5 

3 

4 

W912QR04D00 150028 

5 

4 

4 

5 

4 

3 

5 

4 

5 

3 

3 

4 

4 

5 

3 

5 

W912QR04D00 160009 

5 

5 

3 

4 

4 

4 

5 

3 

0 

3 

3 

0 

0 

3 

0 

0 

W9 1 2QR05C002 1 

5 

5 

5 

4 

5 

4 

5 

5 

4 

4 

5 

5 

4 

4 

3 

5 

W9 1 2QR06C00 1 7 

5 

5 

4 

4 

5 

4 

5 

4 

3 

4 

5 

5 

4 

5 

4 

0 

DAC  A270 1 D00080056 

5 

4 

4 

4 

5 

3 

5 

3 

3 

3 

4 

4 

3 

5 

3 

5 

DAC  A270 1 D00080059 

5 

5 

5 

3 

5 

3 

5 

4 

4 

4 

3 

4 

3 

4 

3 

4 

DACA2701D00 100008 

5 

5 

4 

4 

5 

4 

5 

4 

4 

4 

4 

5 

4 

5 

4 

5 

DACA2703C00 1 8 

5 

5 

5 

4 

5 

4 

5 

5 

3 

4 

4 

5 

3 

4 

4 

0 

W9 1 2QR04D00 1 600 1 7 

5 

5 

4 

4 

4 

3 

5 

5 

3 

3 

3 

5 

4 

5 

3 

5 

W9 1 2QR05C0006 

5 

5 

4 

5 

5 

3 

4 

5 

0 

3 

0 

5 

4 

3 

4 

3 

W912QR04D00 160033 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

W9 1 2QR04D0025000 1 

4 

4 

4 

5 

4 

4 

4 

4 

4 

3 

3 

4 

3 

4 

3 

4 

W9 1 2QR05C0034 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

0 

W9 1 2QR05D00040002 

4 

4 

0 

4 

4 

0 

4 

0 

0 

3 

0 

0 

3 

3 

0 

3 

W9 1 2QR05D00040003 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

0 

W9 1 2QR05D000700 1 7 

3 

3 

3 

3 

3 

3 

3 

3 

3 

0 

0 

3 

3 

3 

0 

0 

W9 1 2QR05D00070020 

4 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

W9 1 2QR05D00070022 

3 

3 

3 

3 

3 

3 

3 

3 

3 

0 

0 

0 

3 

3 

0 

0 

W9 1 2QR05D00070024 

3 

3 

3 

3 

3 

3 

3 

3 

3 

0 

0 

3 

0 

0 

0 

0 

W9 1 2QR05D00070025 

5 

5 

5 

4 

5 

5 

5 

5 

5 

5 

5 

0 

4 

5 

0 

5 

W9 1 2QR06C0020 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

2 

3 

3 

3 

2 

3 

W9 1 2QR06C0057 

3 

3 

3 

3 

3 

3 

4 

3 

3 

3 

3 

3 

2 

3 

2 

3 

DACW2701C0030 

4 

4 

2 

5 

4 

4 

5 

2 

3 

3 

3 

4 

4 

4 

5 

2 

DACW270 1 D000500 1 6 

5 

4 

4 

4 

3 

4 

5 

4 

0 

4 

3 

5 

4 

4 

0 

0 

DACW270 1 D00050087 

3 

4 

3 

3 

3 

4 

4 

4 

0 

3 

3 

3 

3 

3 

0 

0 

DACW270 1 D00050092 

3 

2 

2 

3 

2 

3 

3 

2 

2 

0 

0 

3 

3 

0 

0 

0 

DACW270 1 D00050093 

5 

5 

0 

5 

5 

4 

5 

0 

0 

4 

4 

5 

4 

0 

0 

0 

DACW270 1 D00050094 

5 

4 

4 

5 

4 

5 

5 

0 

0 

0 

4 

0 

0 

0 

0 

0 

DACW270 1 D00050095 

3 

2 

3 

4 

2 

4 

3 

0 

3 

2 

3 

2 

3 

2 

1 

0 

DACW2702C0005 

2 

3 

3 

3 

3 

3 

3 

3 

3 

3 

2 

2 

3 

3 

3 

2 

W912QR04D00 150003 

3 

3 

4 

4 

3 

4 

4 

4 

4 

3 

3 

3 

3 

3 

3 

5 

W912QR04D00 160030 

4 

4 

4 

4 

4 

0 

5 

4 

0 

4 

4 

4 

4 

5 

0 

0 

W9 1 2QR06D000800 1 1 

5 

5 

3 

4 

3 

4 

4 

3 

3 

0 

5 

0 

2 

0 

0 

0 

W9 1 2QR06D00080036 

3 

3 

3 

0 

4 

5 

4 

5 

0 

3 

3 

3 

3 

3 

3 

3 

W9 1 2QR07C004 1 

5 

5 

5 

5 

5 

5 

5 

0 

5 

5 

5 

0 

5 

5 

0 

0 

W9 1 2QR06C00 1 1 

4 

3 

3 

3 

3 

3 

3 

3 

3 

4 

4 

3 

5 

4 

5 

0 

W9 1 2QR06C00 1 6 

4 

3 

3 

3 

3 

3 

4 

3 

3 

3 

5 

3 

5 

4 

5 

0 

DACW2702C00 1 8 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

W9 1 2D06D000 1 C  Y0 1 

5 

3 

4 

3 

4 

3 

4 

3 

3 

3 

3 

5 

3 

3 

3 

0 

W912QR04D00 150030 

4 

4 

3 

3 

4 

4 

4 

3 

3 

4 

4 

4 

3 

3 

3 

3 

W9 1 2QR04C0024 

5 

4 

3 

3 

4 

5 

5 

4 

4 

4 

3 

3 

4 

5 

4 

4 

W912QR04D00 140006 

4 

4 

5 

4 

4 

5 

5 

4 

4 

3 

3 

3 

4 

5 

4 

0 

W912QR04D00 150031 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

W912QR04D00 150033 

3 

3 

3 

3 

4 

3 

3 

4 

3 

3 

4 

3 

3 

3 

3 

3 

W9 1 2QR04D00 1 600 1 6 

4 

4 

2 

5 

4 

3 

4 

3 

2 

3 

3 

3 

3 

5 

4 

5 

W9 1 2QR06C0029 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

DACA2703C0007 

5 

5 

5 

4 

5 

4 

5 

5 

4 

5 

5 

5 

4 

4 

5 

5 

W9 1 2QR06C0039 

4 

4 

3 

4 

3 

3 

3 

4 

3 

3 

3 

4 

3 

3 

3 

3 

W9 1 2QR06C0040 

5 

3 

4 

4 

4 

3 

4 

4 

3 

3 

3 

4 

3 

4 

4 

5 

W9 1 2QR06D000800 1 7 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

W9 1 2QR06D000800 1 8 

5 

4 

4 

0 

4 

3 

5 

0 

0 

0 

0 

0 

3 

0 

0 

0 
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Contract  Number 

18. a 

18.b 

18.c 

19.a 

19.b 

19.c 

DACA2700D00050019 

4 

3 

3 

3 

3 

3 

DACA2700D00050 1 57 

3 

3 

3 

3 

3 

3 

DACA270 1 D00080003 

3 

3 

3 

3 

3 

3 

DACA270 1 D00080040 

3 

3 

3 

3 

3 

3 

DAC  A2702C00 1 3 

3 

3 

3 

3 

2 

3 

DACA2703D0008007 1 

4 

4 

4 

4 

4 

4 

DACA2703D00080072 

3 

3 

3 

3 

3 

3 

DACA2703D00080079 

4 

4 

4 

4 

4 

4 

DACW2700C0016 

4 

3 

3 

3 

3 

3 

DACW2703C0025 

3 

3 

3 

3 

3 

3 

W9 1 2QR04D00 1 600 1 1 

3 

3 

3 

3 

3 

3 

W912QR04D00 160008 

3 

3 

3 

3 

3 

3 

W9 1 2QR06C00 1 0 

3 

3 

3 

3 

5 

4 

W912QR06C0044 

3 

3 

3 

3 

3 

3 

W912QR07C0040 

3 

3 

3 

4 

4 

4 

DAC  A270 1 D00090009 

0 

4 

4 

3 

3 

3 

DACW270 1 D00050029 

3 

3 

3 

3 

3 

3 

W9 1 2QR04D00 150014 

3 

3 

3 

3 

4 

0 

W9 1 2QR04D00 150014 

0 

4 

4 

0 

4 

0 

W912QR05C0013 

3 

3 

3 

3 

3 

3 

W9 1 2QR06D00080040 

3 

3 

3 

3 

3 

3 

W912QR06C0008 

3 

3 

3 

3 

4 

4 

DACW270 1 D00050052 

5 

4 

5 

5 

5 

5 

DACW270 1 D00050078 

3 

3 

3 

3 

3 

3 

W912QR04D00 160027 

3 

3 

3 

3 

3 

3 

W9 1 2QR06D00080007 

0 

3 

3 

3 

3 

3 

W9 1 2QR06D00080032 

0 

4 

3 

3 

3 

3 

W912QR07C001 1 

0 

4 

3 

3 

3 

4 

DACA2701D00 100029 

4 

4 

4 

4 

4 

4 

DAC  A2702C002 1 

2 

2 

2 

3 

2 

2 

DAC  A2703C00 1 0 

4 

4 

4 

4 

5 

5 

DACW270 1 D00050037 

3 

4 

3 

4 

3 

4 

W912QR05C0016 

4 

4 

4 

4 

5 

5 

W912QR05C0020 

4 

4 

4 

3 

3 

4 

W912QR05C0030 

4 

4 

4 

3 

4 

4 

W9 1 2QR04C00 1 2 

5 

4 

4 

3 

3 

4 

W912QR05C0025 

3 

4 

4 

3 

3 

3 

W912QR06C0035 

3 

3 

3 

5 

5 

5 

DACW270 1 D0005000 1 

3 

3 

3 

3 

3 

3 

W912QR06C0014 

3 

3 

3 

4 

4 

4 

DACW270 1 D000501 26 

3 

3 

3 

3 

3 

3 

DACA270 100030059 

4 

3 

3 

3 

3 

4 

DACA270 1 D00030033 

0 

3 

3 

3 

3 

3 

DACA270 1 D00030036 

0 

3 

3 

3 

3 

3 

DACA270 1 D00030040 

3 

3 

3 

3 

4 

4 

DACA270 1 D00030047 

5 

4 

4 

5 

5 

5 

DACA270 1 D00030048 

3 

3 

3 

3 

3 

3 

DACA270 1 D0003005 1 

3 

3 

3 

3 

3 

3 

DACA270 1 D00030054 

0 

4 

4 

5 

5 

5 

DACA270 1 D00030056 

3 

3 

3 

3 

3 

3 

DACA270 1 D00030057 

0 

3 

3 

3 

3 

3 

DACA270 1 D00030058 

3 

3 

3 

3 

3 

3 

DACA270 1 D00030063 

3 

3 

3 

3 

3 

3 

DACA270 1 D00030063 

0 

3 

3 

3 

3 

3 

DACA270 1 D00030065 

3 

3 

3 

4 

4 

4 

Contract  Number 

18.a 

18.b 

18.c 

19.a 

19.b 

19.c 

DACA270 1 D00030069 

3 

3 

3 

3 

3 

3 

DACA270 1 D0003007 1 

3 

3 

3 

3 

3 

3 

DACA270 1 D00030074 

3 

3 

3 

3 

3 

3 

DACA270 1D00030075 

4 

4 

4 

4 

4 

4 

DACA270 1D00030076 

4 

4 

4 

3 

3 

4 

DACA270 1 D00030077 

0 

3 

3 

4 

4 

4 

DACA270 1D00030078 

3 

3 

3 

3 

3 

3 

DACA270 1D0003008 1 

3 

3 

3 

4 

3 

4 

DACA270 1 D0008000 1 

4 

3 

3 

3 

3 

3 

DAC  A27 99D00040004 

4 

5 

5 

5 

4 

4 

DACW270 1 D00050 1 32 

0 

3 

3 

3 

3 

3 

DAC  W270 1 D00050002 

0 

3 

3 

3 

3 

3 

DAC  W270 1 D00050074 

5 

3 

3 

4 

4 

5 

DACW270 1 D00050 1 02 

3 

3 

3 

3 

3 

3 

DACW2702C000 1 

3 

3 

3 

3 

3 

3 

DACW2702C00 1 5 

3 

3 

3 

3 

2 

3 

DACW2703C0008 

4 

3 

3 

3 

5 

4 

DACW2703C0023 

3 

3 

3 

4 

4 

4 

DACW2703C0026 

3 

3 

3 

4 

4 

4 

W912QR04D00 150005 

0 

4 

4 

5 

5 

3 

W912QR04D00 150006 

3 

3 

3 

3 

3 

3 

W912QR04D00 150026 

3 

3 

3 

3 

3 

3 

W912QR04D00 160004 

3 

3 

3 

3 

3 

3 

W9 1 2QR05D0004000 1 

3 

3 

3 

3 

3 

3 

W9 1 2QR05D0007000 1 

3 

3 

3 

3 

3 

3 

W9 1 2QR05D00070002 

3 

3 

3 

4 

4 

4 

W9 1 2QR05D00070006 

3 

3 

3 

4 

4 

4 

W9 1 2QR05D00070008 

3 

3 

3 

3 

3 

3 

W9 1 2QR05D00070009 

3 

3 

3 

3 

3 

3 

W9 1 2QR05D000700 1 1 

3 

3 

3 

3 

3 

3 

W9 1 2QR05D000700 1 2 

3 

3 

3 

3 

3 

3 

W9 1 2QR05D000700 1 3 

3 

3 

3 

3 

3 

3 

DACA2703C0020 

3 

3 

3 

4 

5 

5 

DAC  A2703D0008000 1 

3 

3 

3 

3 

3 

3 

DACA2703D00080003 

0 

4 

4 

4 

4 

4 

DACA2703D00080007 

3 

3 

3 

3 

3 

3 

DACA2703D000800 1 8 

3 

3 

3 

4 

4 

3 

DACA2703D000800 1 9 

4 

4 

4 

4 

4 

4 

DACA2703D00080025 

3 

3 

3 

3 

3 

3 

DACA2703D00080026 

3 

3 

3 

4 

4 

3 

DACA2703D00080029 

3 

3 

3 

3 

4 

3 

DACA2703D00080033 

3 

3 

3 

3 

3 

3 

DACA2703D00080039 

4 

4 

4 

4 

4 

4 

DACA2703D00080040 

3 

3 

3 

3 

3 

3 

DACA2703D00080047 

4 

4 

4 

4 

4 

4 

DACA6303C0020 

3 

3 

3 

3 

4 

4 

DACW270 1 C0022 

3 

3 

3 

3 

3 

3 

DACW270 1 D000500 1 5 

0 

3 

3 

3 

3 

3 

DAC  W270 1 D000500 1 7 

3 

3 

3 

3 

3 

0 

DAC  W270 1 D00050024 

3 

3 

3 

3 

3 

3 

DACW2701D000501 10 

0 

3 

3 

3 

3 

0 

DACW2702C0026 

3 

2 

3 

3 

3 

3 

W9 1 2QR04C0003 

4 

4 

4 

4 

4 

4 

W9 1 2QR04D0009000 1 

3 

3 

3 

3 

3 

3 

W9 1 2OR04D00 150013 

3 

3 

3 

3 

3 

3 

Contract  Number 

18. a 

18.b 

18.c 

19.a 

19.b 

19.c 

W9 1 2QR04D00080003 

4 

3 

3 

4 

4 

3 

DACA270 1 D0008005 8 

0 

3 

3 

3 

4 

3 

W912QR04D00 150007 

5 

4 

4 

4 

4 

5 

W912QR04D00 150022 

5 

3 

3 

4 

5 

5 

W912QR04D00 150028 

5 

4 

3 

4 

5 

5 

W912QR04D00 160009 

3 

3 

3 

3 

4 

4 

W912QR05C0021 

3 

3 

4 

4 

5 

5 

W9 1 2QR06C00 1 7 

0 

4 

4 

4 

3 

5 

DACA270 1 D00080056 

4 

3 

3 

4 

4 

4 

DACA270 1 D00080059 

3 

3 

3 

5 

5 

5 

DACA2701D00 100008 

5 

3 

3 

5 

5 

5 

DAC  A2703C00 1 8 

3 

3 

3 

5 

5 

5 

W9 1 2QR04D00 1 600 1 7 

5 

3 

3 

4 

5 

5 

W912QR05C0006 

4 

3 

3 

4 

3 

5 

W9 1 2QR04D00 1 6003 3 

3 

3 

3 

3 

3 

3 

W9 1 2QR04D0025000 1 

0 

4 

4 

4 

4 

4 

W912QR05C0034 

3 

3 

3 

3 

4 

3 

W9 1 2QR05D00040002 

0 

3 

3 

3 

3 

0 

W9 1 2QR05D00040003 

0 

3 

3 

3 

3 

3 

W9 1 2QR05D000700 1 7 

3 

3 

3 

3 

3 

3 

W9 1 2QR05D00070020 

4 

3 

3 

3 

3 

3 

W9 1 2QR05D00070022 

0 

n 

3 

3 

3 

3 

W9 1 2QR05D00070024 

0 

3 

3 

3 

3 

3 

W9 1 2QR05D00070025 

5 

4 

5 

4 

4 

5 

W9 1 2QR06C0020 

3 

3 

3 

3 

3 

3 

W912QR06C0057 

4 

3 

3 

3 

3 

3 

DACW2701C0030 

4 

4 

4 

4 

4 

4 

DACW270 1 D000500 1 6 

4 

3 

3 

3 

3 

3 

DACW270 1 D00050087 

3 

4 

4 

4 

4 

3 

DACW270 1 D00050092 

3 

3 

3 

3 

3 

3 

DACW270 1 D00050093 

0 

4 

4 

0 

4 

0 

DACW270 1 D00050094 

0 

4 

5 

5 

5 

0 

DACW270 1 D00050095 

3 

3 

4 

4 

4 

4 

DACW2702C0005 

3 

2 

3 

3 

3 

3 

W912QR04D00 150003 

3 

3 

4 

4 

4 

4 

W912QR04D00 160030 

0 

4 

5 

5 

5 

0 

W9 1 2QR06D000800 1 1 

3 

3 

3 

4 

3 

4 

W9 1 2QR06D00080036 

3 

3 

3 

3 

3 

3 

W912QR07C0041 

0 

5 

5 

5 

5 

0 

W9 1 2QR06C00 1 1 

2 

2 

2 

3 

4 

4 

W9 1 2QR06C00 1 6 

2 

2 

2 

3 

4 

4 

DACW2702C00 1 8 

3 

2 

3 

3 

3 

3 

W912D06D0001CY01 

3 

3 

3 

3 

3 

3 

W912QR04D00 150030 

3 

3 

4 

4 

4 

4 

W9 1 2QR04C0024 

4 

5 

5 

5 

5 

5 

W912QR04D00 140006 

4 

3 

5 

5 

4 

4 

W912QR04D00 150031 

3 

3 

3 

3 

3 

3 

W912QR04D00 150033 

3 

3 

3 

3 

3 

3 

W9 1 2QR04D00 1 600 1 6 

4 

4 

4 

4 

5 

4 

W9 1 2QR06C0029 

4 

3 

3 

3 

3 

3 

DACA2703C0007 

4 

4 

4 

5 

5 

5 

W912QR06C0039 

0 

4 

4 

3 

4 

4 

W9 1 2QR06C0040 

4 

3 

4 

4 

5 

5 

W9 1 2QR06D000800 1 7 

3 

3 

3 

3 

3 

3 

W9 1 2QR06D000800 1 8 

0 

3 

3 

3 

3 

0 

Appendix  E:  Normality  Analysis 


Case  Processing  Summary 


Cases 

Valid 

Missing 

Total 

N 

Percent 

N 

Percent 

N 

Percent 

Overall_Rating 

165 

1 00  0% 

0 

.0% 

165 

100  0% 

Descriptives 


Statistic 

Std.  Error 

Overall_Rating  Mean 

3.50 

.054 

95%  Confidence  Interval  Lower  Bound 

3.39 

Upper  Bound 

3.60 

5%  Trimmed  Mean 

3.45 

Median 

3.00 

Variance 

.483 

Std.  Deviation 

.695 

Minimum 

2 

Maximum 

5 

Range 

3 

Interquartile  Range 

1 

Skewness 

.838 

.189 

Kurtosis 

-.188 

.376 

Tests  of  Normality 


a 

Kolmoaorov- Smirnov 

< 

ahaoiro-Wilk 

Statistic 

df 

Sig. 

Statistic 

df 

Sig. 

Overall_Rating 

.357 

165 

.000 

.747 

165 

.000 

a  Lilliefors  Significance  Correction 


Overall_Rating 


Normal  Q-Q  Plot  of  Overall_Rating 
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Descriptives 


Statistic 

Std.  Error 

QC  Mean 

3.1709 

.05128 

95%  Confidence  Interval  Lower  Bound 

3.0697 

for  Mean  Upper  Bound 

3.2722 

5%  Trimmed  Mean 

3.1742 

Median 

3.0000 

Variance 

.434 

Std.  Deviation 

.65871 

Minimum 

1.00 

Maximum 

4.90 

Range 

3.90 

Interquartile  Range 

.70 

Skewness 

.038 

.189 

Kurtosis 

.612 

.376 

Tests  of  Normality 


Kolmoqorov-Smirnov3 

Shapiro-Wilk 

Statistic 

df 

Sig. 

Statistic 

df 

Sig. 

QC 

.161 

165 

.000 

.949 

165 

.000 

a.  Lilliefors  Significance  Correction 


QC 

Normal  Q-Q  Plot  of  QC 
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Descriptives 


Statistic 

Std.  Error 

AAB  Mean 

2.3030 

.11987 

95%  Confidence  Interval  Lower  Bound 

2.0663 

for  Mean  Upper  Bound 

2.5397 

5%  Trimmed  Mean 

2.3165 

Median 

3.0000 

Variance 

2.371 

Std.  Deviation 

1.53981 

Minimum 

.00 

Maximum 

5.00 

Range 

5.00 

Interquartile  Range 

3.00 

Skewness 

-.612 

.189 

Kurtosis 

-1.121 

.376 

Tests  of  Normality 


Kolmoqorov-Smirnov3 

Shapiro-Wilk 

Statistic 

df 

Sig. 

Statistic 

df 

Sig. 

AAB 

.359 

165 

.000 

.763 

165 

.000 

a.  Lilliefors  Significance  Correction 


AAB 


Normal  Q-Q  Plot  of  AAB 
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Descriptives 


Statistic 

Std.  Error 

EM  Mean 

3.5495 

.05026 

95%  Confidence  Interval  Lower  Bound 

3.4503 

for  Mean  Upper  Bound 

3.6487 

5%  Trimmed  Mean 

3.5351 

Median 

3.3333 

Variance 

.417 

Std.  Deviation 

.64556 

Minimum 

ZOO 

Maximum 

5.00 

Range 

3.00 

Interquartile  Range 

1.17 

Skewness 

.446 

.189 

Kurtosis 

-.873 

.376 

Tests  of  Normality 


Kolmoqorov-S  mi  mo  v3 

Shapiro-Wilk 

Statistic 

df 

S*g. 

Statistic 

df 

Sig. 

EM 

.184 

165 

.000 

.912 

165 

.000 

a.  Lilliefbrs  Significance  Correction 


EM 


Normal  Q-Q  Plot  of  EM 
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Descriptives 


Statistic 

Std.  Error 

MS  Mean 

3.0162 

.07372 

95%  Confidence  Interval  Lower  Bound 

2.8706 

for  Mean  Upper  Bound 

3.1617 

5%  Trimmed  Mean 

3.0859 

Median 

3.0000 

Variance 

.897 

Std.  Deviation 

.94697 

Minimum 

.00 

Maximum 

5.00 

Range 

5.00 

Interquartile  Range 

.33 

Skewness 

-1.146 

.189 

Kurtosis 

2.729 

.376 

Tests  of  Normality 


Kolmoqorov-S  mi  mov3 

Shapiro-Wilk 

Statistic 

df 

Sig. 

Statistic 

df 

S*g. 

MS 

.257 

165 

.000 

.854 

165 

.000 

a.  Lilliefors  Significance  Correction 


MS 


Normal  Q-Q  Plot  of  MS 
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Descriptives 


Statistic 

Std.  Error 

TP  Mean 

2.8685 

.07492 

95%  Confidence  Interval  Lower  Bound 

2.7205 

for  Mean  Upper  Bound 

3.0164 

5%  Trimmed  Mean 

2.9091 

Median 

3.0000 

Variance 

.926 

Std.  Deviation 

.96238 

Minimum 

.00 

Maximum 

4.67 

Range 

4.67 

Interquartile  Range 

.78 

Skewness 

-.866 

.189 

Kurtosis 

.633 

.376 

Tests  of  Normality 


Kolmoqorov-Smimov3 

Shapiro-Wilk 

Statistic 

df 

Sig. 

Statistic 

df 

Siq. 

TP 

.245 

165 

.000 

.908 

165 

.000 

a.  Lilliefors  Significance  Correction 


TP 


Normal  Q-Q  Plot  of  TP 
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Descriptives 


Statistic 

Std  Error 

WR  Mean 

2.3091 

.13718 

95%  Confidence  Interval  Lower  Bound 

2.0382 

for  Mean  Upper  Bound 

2.5800 

5%  Trimmed  Mean 

2.2879 

Median 

3.0000 

Variance 

3.105 

Std.  Deviation 

1.76213 

Minimum 

.00 

Maximum 

5.00 

Range 

5.00 

Interquartile  Range 

3.00 

Skewness 

-.251 

.189 

Kurtosis 

-1 .322 

.376 

Tests  of  Normality 


Kolmoqorov-Snmrnov3 

Shapiro-Wilk 

Statistic 

df 

Siq. 

Statistic 

df 

Siq. 

WR 

.295 

165 

.000 

.812 

165 

.000 

a.  Lilliefors  Significance  Correction 


WR 

Normal  Q-Q  Plot  ofWR 
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Descriptives 


Statistic 

Std.  Error 

CLS  Mean 

3.2606 

.04158 

95%  Confidence  Interval  Lower  Bound 

3.1785 

for  Mean  Upper  Bound 

3.3427 

5%  Trimmed  Mean 

3.2306 

Median 

3.0000 

Variance 

.285 

Std.  Deviation 

.53416 

Minimum 

200 

Maximum 

5.00 

Range 

3.00 

Interquartile  Range 

.50 

Skewness 

1.083 

.189 

Kurtosis 

1.240 

.376 

Tests  of  Normality 


Kolmoqorov-Smimov3 

Shapiro-Wilk 

Statistic 

df 

Sig. 

Statistic 

df 

s*g 

CLS 

.402 

165 

.000 

.719 

165 

.000 

a.  Lilliefors  Significance  Correction 


CLS 

Normal  Q-Q  Plot  of  CLS 
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Desmptives 


Statistic 

Sta.  Emor 

CCLSND  Mean 

2.7515 

11123 

35%  Confcence  Interval  Lower  3ound 

2.5310 

for  f/ean  Upper  Bound 

2.0712 

5%  Trmmed  Mean 

2.7795 

Median 

3.0OO0 

Variance 

2.042 

Slid.  Deviation 

1.42882 

Minimum 

.00 

Maximum 

5.00 

Range 

5.00 

Interquartile  Range 

1.00 

SkewTi  ess 

-.950 

.130 

Kurtos  s 

.0152 

.375 

Tests  of  Normality 


Kolmogorov-Smimov3 

Shapiro-Wilh 

Statistic 

df 

Sig. 

S  batiste 

df 

Sig. 

CCLSND 

.353 

165 

ODD 

.760 

155 

.DDQ 

a  Lliefbrs  Sign  licance  Correct  on 


CCLSND 


Normal  Q-Q  Plot  of  CCLSND 
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Desctiptives 


Statist  c 

Std.  Error 

CSS  Mean 

3.4  £54 

.05506 

95%  Confide  nee  Interval  Lower  Eouno 

3.3256 

for  Mean  Upper  Ecund 

3.5469 

5%  Trimmed  Mean 

3.4327 

Median 

3.00DD 

Varance 

.517 

Std.  De^aton 

.71  BID 

Minimum 

1.33 

Ma*  mum 

S.OD 

Range 

3.57 

Interquartle  Range 

1.00 

Skewness 

.313 

.169 

Kuril  asis 

.305 

.375 

T ests  of  Hormal  rty 


Kdmogorov-3  m  mov3 

Shapro-Wilk 

Statstic 

of 

Stg. 

Stat  Stic 

of 

Sig. 

CSS 

.26 1 

165 

.QQQ 

.331 

165 

OQG 

a  Lillielbrs  Significance  Corned  on 


CSS 


Normal  Q-Q  Plot  of  CSS 
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Appendix  F:  Reliability  Analysis 


Performance  Element:  Quality  Control 

Reliability  Statistics 


Cronbach's 

Alpha 

N  of  Items 

.846 

11 

Item-Total  Statistics 


Scale  Mean  if 
Item  Deleted 

Scale 
Variance  if 
Item  Deleted 

Corrected 

Item-Total 

Correlation 

Cronbach's 
Alpha  if  Item 
Deleted 

QC.a 

30.41 

45.340 

.497 

.836 

QC.b 

30.73 

45.285 

.594 

.831 

QC.c 

30.79 

44.461 

.609 

.829 

QC.d 

30.82 

45.451 

.585 

.832 

QC.e 

31.07 

41.312 

.560 

.830 

Item-Total  Statistics 


Scale  Mean  if 
Item  Deleted 

Scale 
Variance  if 
Item  Deleted 

Corrected 

Item-Total 

Correlation 

Cronbach's 
Alpha  if  Item 
Deleted 

QC.f 

30.76 

44.108 

.636 

.827 

QC.g 

30.93 

42.824 

.669 

.823 

QC.h 

31.10 

40.893 

.620 

.825 

QC.i 

31.71 

43.390 

.291 

.866 

QC.j 

30.96 

41.419 

.589 

.827 

QC.k 

30.85 

42.385 

.513 

.835 

Performance  Element:  Effectiveness  of  Management 

Reliability  Statistics 


Cronbach's 

AlDha 

N  of  Items 

.839 

9 

Item-Total  Statistics 


Scale  Mean  if 
Item  Deleted 

Scale 
Variance  if 
Item  Deleted 

Corrected 

Item-Total 

Correlation 

Cronbach's 
Alpha  if  Item 
Deleted 

EM. a 

26.56 

24.029 

.693 

.808 

EM.b 

26.84 

24.321 

.731 

.806 

EM.c 

27.13 

22.929 

.679 

.807 

EM.d 

26.88 

25.798 

.536 

.825 

EM.e 

26.82 

24.406 

.711 

.808 

EM.f 

27.05 

27.271 

.430 

.834 

EM.g 

26.62 

24.760 

.647 

.813 

Item-Total  Statistics 


Scale  Mean  if 
Item  Deleted 

Scale 
Variance  if 
Item  Deleted 

Corrected 

Item-Total 

Correlation 

Cronbach’s 
Alpha  if  Item 
Deleted 

EM.h 

27.18 

24.064 

.499 

.830 

EM.i 

27.68 

25.414 

.278 

.867 
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Performance  Element:  Timely  Performance 


Reliability  Statistics 


Cron  bach's 
Alpha 

N  of  Items 

.810 

7 

Item-Total  Statistics 


Scale  Mean  if 
Item  Deleted 

Scale 
Variance  if 
Item  Deleted 

Corrected 

Item-Total 

Correlation 

Cronbach's 
Alpha  if  Item 
Deleted 

TP. a 

16.72 

29.732 

.782 

.745 

TP.b 

16  69 

31.377 

.649 

.768 

TP.c 

16  54 

31.793 

.564 

.782 

TP.d 

16.60 

33.747 

.586 

.782 

TP.e 

16  39 

34.350 

.487 

.795 

TP.f 

17.20 

29.360 

.626 

.769 

TP -8 

17.25 

33.211 

.286 

.847 

Performance  Element:  Compliance  with  Labor  Standards 

Reliability  Statistics 


Cronbach's 

AlDtia 

N  of  Items 

.268 

3 

Item -Total  Statistics 


Scale  Mean  if 
Item  Deleted 

Scale 
Variance  if 
Item  Deleted 

Corrected 

Item-Total 

Correlation 

Cronbach's 
Alpha  if  item 
Deleted 

CLS.a 

6.52 

1.147 

.020 

.896 

CLS.b 

6.06 

2.426 

.298 

.062 

CLS.c 

6.00 

2.282 

.333 

-013a 

a.  The  value  is  negative  due  to  a  negative  average  covariance  among  items  This  violates  reliability  model  assumptions. 
You  may  want  to  check  item  codings. 


Performance  Element:  Compliance  with  Safety  Standards 

Reliability  Statistics 


Cronbach's 

Alpha 

N  of  hems 

.729 

3 

Item-Total  Statistics 


Scale  Mean  if 
Item  Deleted 

Scale 
Variance  if 
Item  Deleted 

Corrected 

Item-Total 

Correlation 

Cronbach's 
Alpha  if  Item 
Deleted 

CSS. a 

6.90 

2.495 

.663 

543 

CSS.b 

6.76 

2.660 

.571 

.637 

CSS.c 

6.96 

1.889 

.498 

.781 
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Appendix  G:  Principle  Component  Analysis  Results  All  Performance  Items 


Principle  Component  Analysis 

Correlation  Matrix3 

a.  Determinant  =  1.17E-011 


KMO  and  Bartlett's  Test 


Kaiser-Meyer-Olkin  Measure  of  Sampling  Adequacy. 

.856 

Bartlett's  Test  of  Approx.  Chi-Square 

3755.350 

Sphericity  ^ 

528 

Sig. 

.000 

g 

Component  Matrix 


Component 

2 

3 

4 

5 

6 

7 

QC  a 

.600 

-.244 

.077 

.048 

-.127 

.054 

.056 

QCb 

.670 

-.017 

.024 

.350 

.271 

-.354 

-.218 

QC  c 

.736 

-.034 

.117 

.236 

.182 

-.351 

-.104 

QC.d 

.743 

-.161 

-.145 

.017 

.021 

-.040 

-.262 

QC  e 

.432 

.342 

-.008 

.542 

-.305 

.034 

.035 

QC.f 

.610 

.000 

-.327 

.300 

-.100 

-.011 

.234 

QC.fl 

.661 

.117 

-.426 

.210 

-.116 

-.020 

.156 

QC.h 

.551 

.270 

-.120 

.210 

-.132 

.235 

-.040 

QC.i 

.200 

421 

-.240 

.184 

.174 

.422 

-.105 

QCj 

.556 

.224 

-.264 

.271 

-.306 

-.017 

.057 

QC  k 

.508 

.275 

.451 

.010 

-.266 

.082 

-.012 

EM  a 

.600 

-.480 

.057 

-.202 

-.266 

.028 

-.024 

EM.b 

.708 

-.446 

.055 

-.240 

-.161 

-.133 

-.063 

EM.c 

.657 

.123 

.183 

-.122 

.234 

-.354 

.073 

EM.d 

.602 

-.328 

.020 

.031 

-.153 

.124 

.106 

Extraction  Method:  Principal  Component  Analysis, 
a.  7  components  extracted. 
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a 

Component  Matrix 


Component 

1 

2 

3 

4 

5 

0 

7 

EM.e 

.718 

-.437 

.002 

-.133 

-.247 

-.051 

.000 

EM  f 

.513 

-.147 

-.001 

.083 

.303 

.341 

-.111 

EM.g 

.684 

-.522 

.042 

-.148 

-.189 

.043 

-.009 

EM.h 

.004 

.379 

324 

-.180 

.079 

-.074 

.108 

EM.i 

.441 

.477 

-.000 

.002 

.233 

-.111 

.183 

TP. a 

.052 

.280 

-.352 

-.370 

.059 

-.119 

.011 

TP.b 

.038 

.184 

-.374 

-.302 

.191 

-.110 

-.094 

TP.c 

.530 

.289 

-.214 

-.402 

-.249 

.247 

-.113 

TP.d 

.552 

.332 

-.350 

-.210 

-.007 

-.137 

.058 

TP.e 

.010 

.192 

.134 

.028 

-.003 

-.104 

.274 

TP.f 

.430 

.004 

-.127 

-.274 

.080 

.080 

-.290 

TP.g 

.331 

.323 

.422 

-.115 

.007 

.044 

.471 

CLS.a 

.329 

.421 

.407 

-.058 

.035 

.095 

.100 

CLS.b 

.531 

-.412 

-.155 

-.059 

.300 

.370 

.278 

CLS.c 

.595 

-.370 

-.150 

.082 

.500 

.250 

.219 

CSS. a 

.038 

-.109 

.331 

.205 

.219 

-.035 

-.259 

CSS.b 

.725 

-.200 

.152 

-.025 

.038 

.019 

-.154 

CSS.c 

.485 

.249 

544 

-.010 

-.078 

.307 

-.275 

Extraction  Method:  Principal  Component  Analysis, 
a.  7  components  extracted. 


Pattern  Matrix  “ 


Component 

1 

2 

3 

4 

5 

0 

7 

QC.a 

.492 

-.022 

.004 

.220 

.174 

-.110 

.105 

QC.b 

-.020 

.023 

-.049 

.183 

.050 

-.840 

-.052 

QC.c 

.134 

.053 

-.014 

.128 

-.012 

-.772 

.041 

QC.d 

.345 

.280 

.118 

.130 

.134 

-.379 

-.203 

QC.e 

-.091 

-.109 

.137 

.800 

-.153 

-.141 

.110 

QC.f 

.077 

-.015 

-.219 

.701 

.227 

-.094 

.000 

QCg 

.075 

.254 

-.202 

.020 

.104 

-.000 

-.034 

QC.h 

-.014 

.150 

240 

.509 

.119 

-.011 

043 

QCi 

-.455 

.253 

.305 

.277 

.300 

.031 

-.103 

QCj 

.157 

.152 

-.018 

.742 

-.145 

.008 

-.019 

QC  k 

.289 

-.001 

.371 

.224 

-.191 

-.080 

.474 

Extraction  Method:  Principal  Component  Analysis 
Rotation  Method:  ObJimin  with  Kaiser  Normalization. 

a.  Rotation  converged  in  21  iterations. 
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Pattern  Matrix 


Component 

1 

2 

3 

4 

5 

6 

7 

EM  a 

.639 

.093 

.065 

.037 

.091 

-.014 

-.029 

EM.b 

.783 

.177 

-.021 

-.066 

.035 

-.183 

-.023 

EM.c 

.121 

.256 

-.159 

-.119 

.010 

-.486 

.396 

EM  d 

.522 

-.094 

-.020 

.241 

.267 

060 

.125 

EM  e 

.787 

.065 

005 

.095 

.059 

-096 

.008 

EM.f 

.040 

.031 

.271 

-.007 

.555 

-.165 

-.026 

EM.g 

.789 

.039 

.043 

.034 

.178 

-.052 

-.048 

EM  h 

.068 

.272 

.110 

-.046 

-.028 

-.177 

.597 

EM.i 

-.297 

.301 

-.110 

.128 

.101 

-.191 

.432 

TP. a 

.112 

.788 

-.145 

.035 

.062 

-.047 

.100 

TP.b 

.058 

.740 

-.113 

-.034 

.162 

-.187 

-.033 

TP.c 

.281 

.681 

.255 

.113 

-.016 

.302 

.048 

TP.d 

.061 

.648 

-.170 

.223 

-.044 

-005 

.089 

TP.e 

.124 

.094 

-.147 

.226 

.029 

-.166 

.477 

TP.f 

-.202 

.737 

.306 

-.029 

-.082 

-.104 

.089 

TP.g 

-.016 

-,D55 

-.039 

.000 

.112 

.134 

.834 

CLS.a 

-.073 

.002 

.236 

-.005 

-.022 

-.015 

.681 

CLS.b 

.204 

.000 

-.078 

-.050 

.857 

.117 

.089 

CLS.c 

.058 

-.033 

-.118 

-.028 

.870 

-.144 

.069 

CSS. a 

.171 

-.185 

.285 

.009 

.175 

-.647 

.054 

CSS.b 

.464 

.066 

.185 

-.011 

.185 

-.333 

.028 

CSS.c 

.158 

-.017 

.725 

-.014 

-.003 

-.105 

.333 

Extraction  Method:  Principal  Component  Analysis. 
Rotation  Method:  Obtimin  with  Kaiser  Normalization. 

a.  Rotation  converged  in  21  iterations. 


Structure  Matrix 


Component 

1 

2 

3 

4 

5 

6 

7 

QC.a 

.642 

.230 

.139 

.419 

.429 

-.424 

.281 

QC.b 

.295 

.290 

.053 

.432 

.348 

-.890 

.231 

QC.c 

.414 

.320 

.092 

.409 

.316 

-.873 

.324 

QC.d 

.548 

.471 

.182 

.423 

.452 

-.596 

095 

QC.e 

.057 

.151 

.257 

.772 

.032 

-.307 

.291 

QC.f 

.315 

.297 

-.098 

.762 

.435 

-.375 

.173 

QC.g 

.300 

.509 

-.073 

.753 

.400 

-.358 

.180 

QC.h 

.162 

.404 

.346 

.638 

.290 

-.271 

.253 

Extraction  Method:  Principal  Component  Analysis 
Rotation  Method:  Oblimin  with  Kaiser  Normalization. 
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Structure  Matrix 


Component 

2 

3 

4 

5 

6 

7 

QC.i 

-.285 

.363 

.422 

.371 

.283 

-.071 

.037 

QC.j 

.273 

.401 

.097 

.784 

.113 

-.251 

.197 

QC  k 

.375 

.265 

.401 

.423 

.037 

-.358 

.646 

EM  a 

.830 

.251 

.094 

.266 

.398 

-.342 

.151 

EM  b 

.881 

.312 

.016 

.208 

.367 

-.460 

.177 

EM.c 

.350 

.440 

-.027 

.214 

.250 

-.657 

.566 

EM.d 

.642 

.142 

.043 

.383 

.463 

-272 

.233 

EM  e 

.866 

.253 

.051 

.320 

.383 

-.413 

.198 

EM  f 

.275 

.223 

.314 

.227 

.636 

-.374 

.124 

EM.g 

.869 

.210 

.069 

.258 

.466 

-.369 

.124 

EM  h 

.236 

.470 

.258 

.264 

.148 

-.428 

.736 

EM.i 

-.072 

.474 

.039 

.342 

.187 

-.357 

.537 

TP. a 

.280 

.852 

-.023 

.372 

.297 

-.327 

.320 

TP.b 

.263 

.798 

-.012 

.315 

.387 

-.413 

.205 

TP.c 

.311 

.724 

.332 

.367 

.177 

-.026 

.248 

TP.d 

.197 

.732 

-.048 

.455 

.173 

-.251 

.284 

TP.e 

.314 

.347 

.001 

.429 

.224 

-.430 

.595 

TP.f 

-.078 

.763 

.404 

.271 

.064 

-.246 

.321 

TPg 

.094 

.145 

.098 

.149 

.112 

-.119 

.780 

CLS.a 

.027 

.186 

.353 

.166 

.019 

-.198 

.695 

CLS.b 

.447 

.187 

-.033 

.171 

.877 

-.218 

.123 

CLS.c 

.384 

.199 

-.053 

.222 

.918 

-.425 

.147 

CSS. a 

.421 

.091 

.348 

.266 

.411 

-.754 

.285 

CSS.b 

.644 

.282 

.244 

.282 

.463 

-.578 

.256 

CSS.c 

.246 

.196 

.793 

.223 

.136 

-.311 

.510 

Extraction  Method:  Principal  Component  Analysis 
Rotation  Method:  OWimin  with  Kaiser  Normalization. 


Confirmatory  Factor  Analysis  -  5  Components 


Correlation  Matrix3 


a.  Determinant  =  1.17E-011 


KMO  and  Bartlett's  Test 


Kaiser-Meyer-Olkin  Measure  of  Sampling  Adequacy. 

.856 

Bartletfs  Test  of  Approx.  Chi-Square 

3755.350 

Sphericity  _ir 

OT 

528 

Sig 

.000 
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Component  Matrix9 


Comoonent 

1 

2 

3 

4 

5 

QC.a 

.690 

-.244 

.077 

.048 

-.127 

QC.b 

.679 

-.017 

.024 

.350 

.271 

QC.c 

.736 

-.034 

.117 

.236 

.182 

QC.d 

.743 

-.161 

-.145 

.017 

.021 

QCe 

.432 

.342 

-.008 

.542 

-.305 

QCf 

.619 

.000 

-.327 

.390 

-.109 

QC.g 

.661 

.117 

-.426 

.219 

-.116 

QC  h 

.551 

.270 

-.129 

.219 

-.132 

QC.i 

.209 

.421 

-.240 

.184 

.174 

QC.j 

.556 

.224 

-.264 

.271 

-.396 

QC.k 

.598 

.275 

.451 

.019 

-.268 

EM  a 

.690 

-.480 

.057 

-.202 

-.266 

EM  b 

.708 

-.446 

.055 

-.240 

-.161 

EM.c 

.657 

.123 

.183 

-.122 

.234 

EM  d 

.602 

-.328 

.029 

.031 

-.153 

Extraction  Method:  Principal  Component  Analysis, 
a-  5  components  extracted. 


Component  Matrix9 


Component 

1 

2 

3 

4 

5 

EM.e 

.718 

-.437 

.062 

-.133 

-.247 

EM.f 

.513 

-.147 

-.001 

.083 

.303 

EM.g 

.684 

-.522 

.042 

-.148 

-.189 

EM  h 

.604 

.379 

.324 

-.186 

.079 

EM  i 

.441 

.477 

-.006 

.002 

.233 

TP. a 

.652 

.286 

-.352 

-.370 

.059 

TP.b 

.638 

.184 

-.374 

-.302 

.191 

TP.c 

.530 

.289 

-.214 

-.402 

-.249 

TP.d 

.552 

.332 

-.356 

-.216 

-.067 

TP.e 

.610 

.192 

.134 

.028 

-.003 

TP.f 

.430 

.604 

-.127 

-.274 

.086 

TP.g 

.331 

.323 

.422 

-.115 

.067 

CLS.a 

.329 

.421 

.467 

-.058 

.035 

CLS.b 

.531 

-.412 

-.155 

-.059 

.360 

CLS.c 

.595 

-.370 

-.150 

.082 

.506 

CSS. a 

.638 

-.169 

.331 

.265 

.219 

CSS.b 

.725 

-.266 

.152 

-.025 

.038 

CSS.c 

.485 

.249 

.544 

-.010 

-.078 

Extraction  Method:  Principal  Component  Analysis, 
a.  5  components  extracted. 
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Pattern  Matrix3 


Component 

1 

2 

3 

4 

5 

QC.a 

.513 

.005 

.135 

.223 

.185 

QC  b 

-.011 

-.062 

.164 

.345 

.603 

QC.c 

.133 

-.054 

.268 

.274 

.499 

QC.d 

.364 

.216 

-.007 

.225 

.329 

QC  e 

-.081 

-.167 

.181 

.842 

-.098 

QC.f 

.118 

.065 

-.181 

.681 

.235 

QC.g 

.106 

.316 

-.184 

.608 

.161 

QC.h 

.010 

.168 

.120 

.526 

.043 

QC.i 

-.421 

.238 

.013 

.309 

.186 

QC.j 

.167 

.181 

-.043 

.744 

-.180 

QC.k 

.267 

-.018 

.691 

.249 

-.173 

Extraction  Method:  Principal  Component  Analysis- 
Rotation  Method:  Oblimin  with  Kaiser  Normalization. 

a.  Rotation  converged  in  15  iterations. 


Pattem  Matrix3 


Component 

1 

2 

3 

4 

5 

EM. a 

.852 

.110 

.035 

.028 

.043 

EM.b 

.785 

.163 

.063 

-.046 

.132 

EM.c 

.102 

.227 

.432 

-.070 

.364 

EM  d 

.552 

-.012 

.028 

.193 

.153 

EM.e 

.797 

.079 

.058 

.095 

.079 

EM.f 

.089 

.033 

.073 

.024 

.541 

EM.g 

.809 

.061 

-004 

.028 

.148 

EM  h 

.043 

.275 

.667 

-.043 

.087 

EM.i 

-.303 

.326 

.345 

.123 

.234 

TP.a 

.105 

.802 

.037 

.020 

.087 

TP.b 

.061 

.723 

-.037 

-.014 

.277 

TP.c 

.276 

.668 

.124 

.074 

-.268 

TP.d 

.053 

.671 

.001 

.203 

-.028 

TP.e 

.118 

.146 

.375 

.204 

.141 

TP.f 

-.223 

.660 

.320 

.028 

-.030 

TP -9 

-.030 

.060 

.648 

-.068 

.017 

CLS.a 

-.064 

.027 

.724 

-.023 

-.031 

CLS.b 

.275 

.135 

-.165 

-.138 

.649 

CLS.c 

.130 

.064 

-.136 

-.066 

.845 

CSS.a 

.187 

-.288 

.375 

.149 

.539 

CSS.b 

.479 

.024 

.220 

.053 

.340 

CSS.c 

.150 

-.087 

.747 

.055 

-.030 

Extraction  Method:  Principal  Component  Analysis. 
Rotation  Method:  Ob  im  n  with  Kaiser  Normalization. 

a.  Rotation  converged  in  15  iterations. 
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Structure  Matrix 


Component 

1 

2 

3 

4 

5 

QC.a 

.647 

.242 

.321 

.428 

.474 

QCb 

.293 

.238 

.376 

.548 

.717 

QC.c 

.405 

.270 

.468 

.520 

.677 

QC.d 

.560 

.436 

.248 

.478 

.591 

QC.e 

.058 

.146 

.365 

.791 

.126 

QC.f 

.325 

.337 

.108 

.742 

.466 

QC.g 

.304 

.540 

.132 

.733 

.432 

QC.h 

.174 

.402 

.341 

.639 

.282 

Extraction  Method:  Principal  Component  Analysis. 
Rotation  Method:  Obtimin  with  Kaiser  Normalization. 


Structure  Matrix 


Component 

1 

2 

3 

4 

5 

QC.i 

-.255 

.347 

.153 

.370 

.194 

QC.j 

.271 

.410 

.223 

.774 

.152 

QC.k 

.353 

.248 

.765 

.459 

.149 

EM  a 

.893 

.258 

.207 

.266 

.404 

EM.b 

.855 

.306 

.236 

.233 

.465 

EM  c 

.316 

.431 

566 

.280 

.535 

EM.d 

.650 

.181 

.199 

.357 

.419 

EM  e 

.865 

.259 

.243 

.329 

.437 

EM.f 

.308 

.219 

.220 

.245 

.606 

EM.g 

.877 

.220 

.171 

.259 

.471 

EM  h 

.201 

.470 

.753 

.300 

.309 

EM.i 

-.096 

.489 

.480 

.360 

.325 

TP. a 

.259 

.860 

.297 

.370 

.367 

TP.b 

.254 

.792 

.223 

.333 

.483 

TP.c 

.305 

.723 

.318 

.336 

.073 

TP.d 

.176 

.744 

.248 

.446 

.237 

TP.e 

.288 

.375 

.526 

.441 

.369 

TP.f 

-.091 

.718 

.468 

.310 

.145 

TP.g 

.059 

.201 

.633 

.124 

.133 

CLS.a 

.000 

.194 

.704 

.184 

.092 

CLS.b 

.481 

.254 

.010 

.116 

.709 

CLS.c 

.418 

.250 

064 

.203 

.860 

CSS. a 

.432 

.039 

489 

.368 

.658 

CSS.b 

.651 

.259 

.387 

.333 

.588 

CSS.c 

.248 

.147 

.756 

.278 

.182 

Extraction  Method:  Principal  Component  Analysis. 
Rotation  Method:  OWimin  with  Kaiser  Normalization. 


152 


Appendix  H:  Principle  Component  Analysis  Results  for  Quality  Control 


Initial  PCA 


a 

Correlation  Matrix 


QC.d 

QC.e 

QC.f 

QC.g 

QC.h 

QC.i 

QC i 

QC.k 

Correlation  QC.d 

1.000 

.213 

.419 

.522 

.406 

.130 

.400 

.322 

QC.e 

.213 

1.000 

.448 

.358 

.442 

.220 

.520 

440 

QC.f 

.410 

.448 

1.000 

.652 

.402 

.173 

.549 

.255 

QC.g 

.522 

.358 

.652 

1.000 

.510 

.307 

.583 

.281 

QC.h 

.406 

.442 

.402 

.510 

1.000 

.400 

.334 

.425 

QC.i 

.130 

.229 

.173 

.307 

.400 

1.000 

.174 

.214 

QCj 

.400 

.520 

.549 

.583 

.334 

.174 

1.000 

.325 

QC.k 

.322 

.440 

.255 

.281 

.425 

.214 

.325 

1.000 

QC.c 

.640 

.309 

.397 

446 

.346 

.117 

.355 

.435 

QC.b 

.558 

.339 

.485 

411 

.332 

.129 

.381 

.334 

QC.a 

496 

.302 

.501 

.338 

.366 

.003 

.326 

349 

a.  Determinant  =  .006 


Correlation  Matrix3 


QC.c 

QC.b 

QC.a 

Correlation  QC.d 

640 

.558 

.496 

QC.e 

.309 

.339 

.302 

QC.f 

.307 

.485 

.501 

QC.g 

446 

.411 

.338 

QC.h 

.346 

.332 

.366 

QC.i 

.117 

.129 

.003 

QCj 

.355 

.381 

.326 

QC.k 

.435 

.334 

.340 

QC.c 

1.000 

.791 

.465 

QC.b 

.791 

1.000 

.403 

QC.a 

465 

.403 

1.000 

a.  Determinant  =  .006 


KMO  and  Bartlett's  Test 


Kaiser-Meyer-Olkin  Measure  of  Sampling  Adequacy. 

.821 

Bartlett's  Test  of  Approx.  Chi-Square 

815.498 

Sphericity  ^ 

55 

Sig. 

.000 
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Component  Matrix3 


Component 

1 

2 

QC.d 

.723 

-.344 

QC.e 

.619 

.365 

QC.f 

.743 

.044 

QC.g 

.754 

.173 

QC.h 

.666 

.366 

QC.i 

.331 

.650 

QC.j 

.688 

.179 

QCk 

.584 

.131 

QC.c 

.751 

-.430 

OCb 

.732 

-.385 

QC  a 

.639 

-.303 

Extraction  Method:  Principal  Component  Analysis, 
a.  2  components  extracted. 


Confirmatory  PCA 


Quality  Control  1  (QC1) 


Correlation  Matrix* 


QC.d 

QC.f 

QC.a 

QC  It 

QCk 

QC.c 

QC.b 

QC  a 

Correlation  QC.d 

1  000 

.419 

.522 

406 

.322 

640 

.558 

.496 

QC.f 

.419 

1  000 

.652 

.402 

255 

.397 

.485 

501 

QC.g 

.522 

,652 

1  000 

.510 

281 

.446 

.411 

338 

QC.h 

.406 

402 

.510 

1000 

.425 

.346 

.332 

.366 

QC.k 

.322 

255 

281 

.425 

1  000 

.435 

.334 

349 

QC.c 

.640 

.397 

.446 

.346 

.435 

1  000 

.791 

465 

QC.b 

.568 

.485 

.411 

332 

.334 

.791 

1.000 

403 

QC.a 

.496 

.501 

.338 

366 

.349 

.465 

.403 

1000 

a.  Determinant  =  023 


KMO  and  Bartlett's  Test 


Kaiser-Meyer-Olkin  Measure  of  Sampling  Adequacy 

.785 

Bartiett's  Test  of  Approx.  Ch  -Square 

602.051 

Sphencity  a 

28 

Sig. 

000 

Component  Matrix* 


Component 

1 

QC.d 

.777 

QC.f 

.722 

QC  g 

.731 

QCh 

645 

QC.k 

567 

QC.c 

807 

QC.b 

772 

QC  a 

.681 

Extraction  Method:  Prncipal  Component  Analyss 
a.  1  components  extracted. 
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Quality  Control  2  (QC2) 


a 

Correlation  Matrix 


a.  Determinant  =  .688 

KMO  and  Bartlett's  Test 


Kaiser-Meyer-Olkin  Measure  of  Sampling  Adequacy. 


.555 

Bartlett's  Test  of 

Approx.  Chi-Square 

60.553 

Sphericity 

df 

3 

Sig 

.000 

a 

Component  Matrix 


Component 

1 

QC.i 

.518 

QC.j 

.817 

QC.e 

.842 

Extraction  Method:  Principal  Component  Analysis, 
a.  1  components  extracted. 
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Appendix  I:  Principle  Component  Analysis  Results  for  Effectiveness  of  Management 


Initial  PCA 


a 

Correlation  Matrix 


EM.a 

EM.b 

EM.c 

EM.d 

EM.e 

EM.f 

EM.g 

Correlation  EM.a 

1.000 

.781 

.401 

.573 

.787 

.365 

.815 

EM.b 

.781 

1.000 

.482 

.522 

.788 

.335 

.743 

EM.c 

.401 

.482 

1.000 

.283 

.422 

.346 

.358 

EM.d 

.573 

.522 

.283 

1.000 

.561 

.274 

.534 

EM.e 

.787 

.788 

.422 

.561 

1.000 

.333 

.772 

EM.f 

.365 

.335 

.346 

.274 

.333 

1.000 

.380 

EM.g 

.815 

.743 

.358 

.534 

.772 

.380 

1.000 

EM.h 

.237 

.351 

.710 

.181 

.277 

.238 

.210 

EM.i 

.061 

.105 

.480 

.162 

.088 

.175 

-.002 

a.  Determinant  =  .004 


KMO  and  Bartlett's  Test 


Kaiser-Meyer-Olkin  Measure  of  Sampling  Adequacy. 


.868 

Bartletfs  Test  of 

Approx.  Chi-Square 

874.682 

Sphericity 

df 

36 

Sig. 

000 

a 

Component  Matrix 


Component 

1 

2 

EM.a 

.872 

-.281 

EM.b 

.875 

-.162 

EM.c 

.661 

.603 

EM.d 

.678 

-.163 

EM.e 

.872 

-.238 

EM.f 

.518 

.118 

EM.g 

.842 

-.337 

EM.h 

.505 

.672 

EM.i 

.272 

.732 

Extraction  Method:  Principal  Component  Analysis 
a.  2  components  extracted. 
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Confirmatory  PCA 


Effectiveness  of  Management 

■ 

Correlation  Matrix 


EM  a 

EM.b 

EM.d 

EM.e 

EM.f 

EM.g 

Correlation  EM.a 

1.000 

.781 

.573 

.787 

.365 

.815 

EM.b 

.781 

1.000 

.522 

.788 

.335 

.743 

EM.d 

.573 

.522 

1.000 

.561 

.274 

.534 

EM.e 

.787 

.798 

.561 

1.000 

.333 

.772 

EM.f 

.365 

.335 

.274 

.333 

1.000 

.390 

EM.g 

.815 

.743 

.534 

.772 

.380 

1.000 

a.  Determinant  =  .017 


KMO  and  Bartlett's  Test 

Kaiser-Meyer-Olkin  Measure  of  Sampling  Adequacy. 


.888 

Bartlett's  Test  of 

Approx.  Chi-Square 

655.767 

Sphericity 

df 

15.000 

Sig. 

.000 

Component  Matrix3 


Component 

1 

EM  a 

.915 

EM.b 

.884 

EM.d 

.707 

EMe 

.902 

EM.f 

.495 

EM.g 

.896 

Extraction  Method:  Principal  Component  Analysis, 
a.  1  components  extracted. 
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Management  of  Subcontractors 


a 

Correlation  Matrix 


EM.c 

EM.h 

EM.i 

Correlation  EM.c 

1.000 

.710 

.480 

EM.h 

.710 

1.000 

.397 

EMJ 

480 

.387 

1.000 

a-  Determinant  =  .378 


KMO  and  Bartlett's  Test 


Kaiser-Meyer-Olkin  Measure  of  Sampling  Adequacy. 

.632 

Bartlett's  Test  of  Approx.  Chi-Square 

157.562 

Sphericity  ^ 

3.000 

Sig. 

000 

a 

Component  Matrix 


Component 

1 

EM.c 

.887 

EM.h 

.863 

Extraction  Method:  Principal  Component  Analysis, 
a.  1  components  extracted. 
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Appendix  J:  Principle  Component  Analysis  Results  for  Timely  Performance 


Initial  PCA 


Correlation  Matrix* 


TP.a 

TP.b 

TP.c 

TP.d 

TP.e 

TP.f 

TPg 

Correlation  TP.a 

1.000 

.788 

.572 

.501 

.465 

.582 

.255 

TP.b 

.788 

1000 

.483 

502 

.372 

.514 

.137 

TP.c 

.572 

483 

1  000 

.454 

.248 

.517 

.164 

TP.d 

.501 

.502 

.454 

1  000 

.382 

499 

.121 

TP.e 

.465 

.372 

248 

382 

1  000 

281 

.353 

TP.f 

.582 

.514 

.517 

499 

281 

1  000 

263 

TPg 

.255 

.137 

.164 

.121 

.353 

263 

1  000 

a.  Deteiminant  =  .056 


KMO  and  Bartlett's  Test 


Kaiser-Meyer-Olk  n  Measure  of  Sampling  Adequacy 

.824 

BartieTt’s  Test  of  Approx  Ch  -Square 

451.758 

Sphenaty 

21 

Sig. 

000 

Component  Matrix* 


Component 

1 

2 

TP.a 

893 

-084 

TP.b 

.810 

-204 

TP.c 

.715 

-230 

TP.d 

.742 

-.174 

TP.e 

,592 

496 

TP.f 

.755 

-085 

TPg 

.371 

.819 

Extraction  Method:  Prncipal  Component  Analyss 
a.  2  components  extracted. 
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Confirmatory  PCA 


Correlation  Matrix3 


TP.a 

TP.b 

TP.c 

TP.d 

TP.e 

TP.f 

Correlation  TP. a 

1.000 

.788 

.572 

.591 

.405 

.582 

TP.b 

.788 

1.000 

.483 

.502 

.372 

.514 

TP.c 

.572 

.483 

1.000 

.454 

.248 

.517 

TP.d 

.591 

.502 

.454 

1.000 

.382 

.499 

TP.e 

405 

.372 

.248 

.382 

1.000 

.281 

TP.f 

582 

.514 

.517 

499 

.281 

1.000 

a  Determinant  =  .071 


KMO  and  Bartlett's  Test 


Kaiser-Meyer-Olkin  Measure  of  Sampling  Adequacy. 

.842 

Bartlett's  Test  of  Approx.  Chi-Square 

421.040 

Sphericity 

15.000 

Sig 

000 

a 

Component  Matrix 


Component 

1 

TP.a 

.897 

TP.b 

.824 

TP.c 

.723 

TP.d 

.750 

TP.e 

.509 

TP.f 

.753 

Extraction  Method:  Principal  Component  Analysis, 
a.  1  components  extracted. 
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Appendix  K:  Principle  Component  Analysis  Results  for  Compliance  with  Labor 

Standards 


Initial  PCA 


i 

Correlation  Matrix 


CLS.a 

CLS.b 

CLS.c 

Correlation  CLS.a 

1.000 

-.010 

.045 

CLS.b 

-.010 

1.000 

.815 

CLS.c 

.045 

.815 

1.000 

a-  Determinant  =  .333 


KMO  and  Bartlett's  Test 


Kaiser-Meyer-Olkin  Measure  of  Sampling  Adequacy. 


494 

Bartlett's  Test  of 

Approx.  Chi-Square 

177.220 

Sphericity 

df 

3 

Sig. 

.000 

a 

Component  Matrix 


Component 

1 

2 

CLS.a 

.042 

.999 

CLS.b 

.952 

-.055 

CLS.c 

.953 

.012 

Extraction  Method:  Principal  Component  Analysis, 
a.  2  components  extracted. 


Confirmatory  PCA 


Correlation  Matrix* 


CLS.b 

CLS.c 

Correlation  CLS.b 

CLS.c 

1  000 

.815 

.815 

1.000 

a.  Determinant  =  .336 


KMO  and  Bartlett's  Test 


Kaiser-Meyer-Olkin  Measure  of  Sampling  Adequacy 

500 

Bartlett's  Test  of  Approx.  Cb-Square 

176211 

Sphencity  ^ 

1  000 

Sig. 

.000 

Component  Matrix* 


Component 

1 

CLS.b 

.953 

CLS.c 

.953 

Extraction  Method:  Prncipal  Component  Analyse 
a.  1  components  extracted. 
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Appendix  L:  Principle  Component  Analysis  Results  for  Compliance  with  Safety 

Standards 


Initial  PCA 

Correlation  Matrix1 


CSSa 

CSSb 

CSS.c 

Correlation  CSS.a 

1  000 

.641 

503 

CSSb 

641 

1.000 

400 

CSS.c 

503 

.400 

1.000 

a.  Determinant  =  .434 


KMO  and  Bartlett's  Test 


Kaiser-Meyer-Olk  n  Measure  of  Sampling  Adequacy 

.650 

Bartlett's  Test  of  Approx  Ch  -Square 

135.301 

Sphericity  ^ 

3 

Sig. 

000 

Component  Matrix9 


Component 

1 

CSSa 

.881 

CSSb 

.835 

Extraction  Method:  Principal  Component  Analyss 
a.  1  components  extracted. 


Confirmatory  PCA-  Not  Required 
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Appendix  M:  Performance  Element  Correlation  Analysis  Results 


Convtattortc 


Overai_ 

fiatrg 

QC1 

OC2 

EM 

MS 

TP 

Overai_Ratng 

®ear«r  ConettBon 

1  COO 

629 

219~ 

969 

39C 

449 

SJg.  tMaiedi 

OX 

X5 

OX 

xo 

XC 

N 

165 

165 

165 

165 

165 

165 

QC1 

»ear5or  Coereaton 

.626 

1.0CC 

6C5 

566 

073 

625 

SJg.  'Xaieai 

.000 

.003 

OX 

X0 

OX 

N 

165 

165 

165 

165 

165 

165 

QC2 

^earsor  Cofreaton 

.219" 

6D5~ 

tooo 

222" 

.390 " 

471" 

SJg.  (Maledi 
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OX 
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XO 

OX- 

N 
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155 

165 

165 

165 

EM 
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666” 

222~ 
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.000 

OX 
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XO 

OX 

N 
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MS 
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390 

349 

1  OX 

588 
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.000 

OX 
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OX 

OX 

N 
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165 

165 

165 

TP 

searsor  ConeMttn 

.4*9 
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.471 

.405 

596 

1  XO 

SJg.  tXaiedi 

3X 

OX 

.003 

OX 

XO 

N 
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165 
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165 

165 

165 

AR 

Parser  Cofreaton 
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294" 

.194' 
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414" 

274" 
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.211 

OX 
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XO 

OX 

N 
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CCLSND 

Parser  Cotreaton 
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-213" 

-its 

436" 
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OX 
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XO 

.OX 

N 
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155 

165 

165 

165 

CL S 

^earsor  ConcMBn 
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455" 

.120 

.578 

259~ 

.294 

SJg.  •  2-taieci 

.000 

OX 

.124 

OX 

001 

.XC 

N 

165 

165 

155 

165 

165 

165 

CSS 

^earsor  Cones  aeon 

.535 

618 

325 

.576 

457 

.414 

SJg.  (>ta»edi 

ox 

OX 

.003 

OX 

000 

OX 

N 

165 

165 

155 

165 

165 

165 

•  Coreiaoon  e  sgntcant  at  me  OOi  leve  i2-ttied> 
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OX 
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^earsor  Coneiadon 

.194’ 
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013 

.006 

124 

XO 

N 
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EM 
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.137 
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OX 
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414’ 

SJg.  -Xaiedi 

OX 

X3 

OX 

XC 

N 
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*65 

AR 
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1  OX 

47l" 

134 

304- 

SJg.  -Maiedi 

X3 
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XC 

N 
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155 

165 

CCLSND 

^earwr  Correlator 

471  ~ 

lOX 

025 

.387" 

SJg.  tMaiedi 

OX 

746 

XC 

N 

165 

165 

165 

165 

CLS 

®ear3or  Correlator 

.134 

.025 

1.0X 

.393" 

SJg.  totaled 1 

.396 

.746 

XC 

N 

165 

155 

i: 

•65 

CSS 

^earsor  Correlator 

304" 

397" 

.393" 

10X 

SJg.  -;2ta»edi 
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OX 

OX 
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155 
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Appendix  N:  Step-Wise  Linear  Regression  Results 


Variables  Entered/Removed1 


Mode 

l 

Variables 

Entered 

Variables 

Removed 

Method 

1 

EM 

Stepwise 
(Criteria: 
Probability-of- 
F-to-enter  <= . 
050, 

Probability-of- 

F-to-remove 

>=.100). 

2 

TP 

Stepwise 
(Criteria: 
Probability-of- 
F-to-enter  <= . 
050, 

Probability-of- 
F-to-remove 
>=  .100). 

a.  Dependent  Variable:  Overall_Rating 


Model  Summary0 


Mode 

l 

R 

R  Sauare 

Adjusted  R 
Sauare 

Std.  Error  of 
the  Estimate 

Durbin- 

Watson 

1 

,869a 

.755 

.753 

.345 

2 

875b 

.766 

.763 

.338 

1.884 

a.  Predictors:  (Constant),  EM 

b.  Predictors:  (Constant),  EM,  TP 

c.  Dependent  Variable:  Overall_Rating 


ANOVAC 


Model 

Sum  of 
Squares 

df 

Mean  Square 

F 

Siq. 

1 

Regression 

59.821 

1 

59.821 

501.904 

,oooa 

Residual 

19.428 

163 

.119 

Total 

79.248 

164 

2 

Regression 

60.698 

2 

30.349 

265  042 

.ooob 

Residual 

18.550 

162 

.115 

Total 

79.248 

164 

a.  Predictors:  (Constant),  EM 
b  Predictors:  (Constant),  EM,  TP 
c.  Dependent  Variable:  Overall_Rating 
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Coefficients' 


Model 

Unstandardized  Coefficients 

Standardized 

Coefficients 

t 

Sid. 

95%  Confidence  Interval  for  B 

B 

Std.  Error 

Beta 

Lower  Bound 

Uooer  Bound 

1 

(Constant) 

.176 

.151 

1.170 

.244 

-.121 

.474 

EM 

.936 

.042 

.869 

22.403 

.000 

.853 

1.018 

2 

(Constant) 

.116 

.149 

.777 

.438 

-.179 

.411 

EM 

.885 

.045 

.822 

19.774 

.000 

.797 

.974 

TP 

.083 

.030 

.115 

2.768 

.006 

.024 

.142 

a  Dependent  Variable:  OverallRating 


Coefficients'1 


Model 

Collinearitv  Statistics 

Tolerance 

VIF 

1  (Constant) 

EM 

1.000 

1.000 

2  (Constant) 

EM 

TP 

.836 

.836 

1.197 

1.197 

a.  Dependent  Variable:  Overall_Rating 


Excluded  Variables0 


Model 

Beta  In 

t 

Sia. 

Partial 

Correlation 

Collinearitv  Statistics 

Tolerance 

VIF 

Minimum 

Tolerance 

1  QC 

,068a 

1.389 

.167 

.108 

.624 

1.602 

.624 

MS 

088a 

2.147 

.033 

.166 

.879 

1.138 

.879 

TP 

,115a 

2.768 

.006 

.213 

.836 

1.197 

.836 

CCLSND 

,057a 

1.460 

.146 

.114 

.987 

1.013 

.987 

CLS 

038a 

.795 

.428 

.062 

.666 

1.503 

.666 

CSS 

,053a 

1.110 

.269 

.087 

.669 

1.496 

.669 

2  QC 

ooob 

-.014 

.989 

-.001 

.459 

2.179 

.459 

MS 

040b 

.848 

.398 

.067 

640 

1.563 

.609 

CCLSND 

.031 b 

.775 

.440 

.061 

.917 

1.091 

.777 

CLS 

029b 

.629 

.530 

.049 

.663 

1.509 

.602 

CSS 

023b 

.476 

.635 

.037 

.630 

1.588 

.630 

a.  Predictors  in  the  Model:  (Constant),  EM 

b.  Predictors  in  the  Model:  (Constant),  EM,  TP 

c.  Dependent  Variable:  Overall  Rating 
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Coefficient  Correlations3 


Model 

EM 

TP 

1  Correlations  EM 

1.000 

Covariances  EM 

.002 

2  Correlations  EM 

1.000 

-.405 

TP 

-.405 

1.000 

Covariances  EM 

.002 

.000 

TP 

.000 

.001 

a.  Dependent  Variable:  Overall_Rating 


Coilinearity  Diagnostics 


Mode 

1 

Dime 

nsio 

n 

Eigenvalue 

Condition 

Index 

Variance  Proportions 

(Constant) 

EM 

TP 

1 

1 

1.984 

1.000 

.01 

.01 

2 

.016 

11.120 

.99 

.99 

2 

1 

2.926 

1.000 

.00 

.00 

.01 

2 

.059 

7.056 

.12 

.05 

.95 

3 

.016 

13.707 

.88 

.95 

.04 

a.  Dependent  Variable:  Overall_Rating 


Residuals  Statistics3 


Minimum 

Maximum 

Mean 

Std.  Deviation 

N 

Predicted  Value 

2.07 

4.82 

3.50 

.608 

165 

Residual 

-1.216 

1.361 

.000 

.336 

165 

Std.  Predicted  Value 

-2.351 

2.174 

.000 

1.000 

165 

Std.  Residual 

-3.593 

4.021 

.000 

.994 

165 

a.  Dependent  Variable:  Overall_Rating 


Dependent  Variable:  Overall_Rating 


Regression  Standardized  Residual 


Mean  =4.28E-16 
Std.  Dev.  =0.994 
N  =165 
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Regression  Standardized  Residual  Expected  Cum  Prob 


Normal  P-P  Plot  of  Regression  Standardized  Residual 


Dependent  Variable:  Overall_Rating 


Scatterplot 


Dependent  Variable:  Overall_Rating 


Regression  Standardized  Predicted  Value 


167 


Appendix  O:  Principle  Component  Analysis  Results  of  Reversed  Scored  CLS 


g 

Component  Matrix 


Component 

1 

CLS. a 

.669 

CLS.b 

.925 

Extraction  Method:  Principal  Component  Analysis, 
a.  1  components  extracted. 

g 

Component  Matrix 


Component 

1 

CLS.c 

.901 

Extraction  Method:  Principal  Component  Analysis, 
a.  1  components  extracted. 
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Appendix  P:  EM  Performance  Items  Correlation  Analysis  Results 


Correlations 


Overall_ 

Ratina 

EM. a 

EM.b 

EM.d 

EM.e 

Overall_Rating 

Pearson  Correlation 

1.000 

.796" 

.758" 

.556" 

.801 " 

Sig.  (2-tailed) 

.000 

.000 

.000 

.000 

N 

165.000 

165 

165 

165 

165 

EM. a 

Pearson  Correlation 

.796" 

1.000 

.781 " 

.573" 

.797" 

Sig.  (2-tailed) 

.000 

.000 

.000 

.000 

N 

165 

165.000 

165 

165 

165 

EM.b 

Pearson  Correlation 

.758" 

.781 " 

1.000 

.522” 

.798” 

Sig.  (2-tailed) 

.000 

.000 

.000 

.000 

N 

165 

165 

165.000 

165 

165 

EM.d 

Pearson  Correlation 

.556” 

.573" 

.522" 

1.000 

.561 " 

Sig.  (2-tailed) 

.000 

.000 

.000 

.000 

N 

165 

165 

165 

165.000 

165 

EM.e 

Pearson  Correlation 

.801 " 

.797” 

.798" 

.561 " 

1.000 

Sig.  (2-tailed) 

.000 

.000 

.000 

.000 

N 

165 

165 

165 

165 

165.000 

EM.f 

Pearson  Correlation 

.485” 

.365" 

.335" 

.274" 

.333" 

Sig.  (2-tailed) 

.000 

.000 

.000 

.000 

.000 

N 

165 

165 

165 

165 

165 

EM.g 

Pearson  Correlation 

.787" 

.815" 

.743" 

.534" 

112 

Sig.  (2-tailed) 

.000 

.000 

.000 

.000 

.000 

N 

165 

165 

165 

165 

165 

Correlation  is  significant  at  the  0.01  level  (2-tailed). 


Correlations 


EM.f 

EM. a 

Overall_Rating 

Pearson  Correlation 

.485" 

.787" 

Sig.  (2-tailed) 

.000 

.000 

N 

165 

165 

EM. a 

Pearson  Correlation 

.365” 

.815*’ 

Sig.  (2-tailed) 

.000 

.000 

N 

165 

165 

EM.b 

Pearson  Correlation 

.335" 

.743” 

Sig.  (2-tailed) 

.000 

.000 

N 

165 

165 

EM.d 

Pearson  Correlation 

.27 4" 

.534" 

Sig.  (2-tailed) 

.000 

.000 

N 

165 

165 

EM.e 

Pearson  Correlation 

.333” 

112 

Sig.  (2-tailed) 

.000 

.000 

N 

165 

165 

EM.f 

Pearson  Correlation 

1.000 

.390*’ 

Sig.  (2-tailed) 

.000 

N 

165.000 

165 

EM.g 

Pearson  Correlation 

,390” 

1.000 

Sig.  (2-tailed) 

.000 

N 

165 

165.000 

**.  Correlation  is  significant  at  the  0.01  level  (2-tailed). 
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Appendix  Q:  EM  Performance  Items  Linear  Regression  Results 


Variables  Entered/Removed3 


Mode 

1 

Variables 

Entered 

Variables 

Removed 

Method 

i 

EM.e 

Stepwise 
(Criteria: 
Probability-of- 
F-to-enter  <=  . 
050. 

Probability-of- 
F-to-remove 
>=  .100). 

2 

EM.g 

Stepwise 
(Criteria: 
Probability-of- 
F-to-enter  <=  . 
050. 

Probability-of- 

F-to-remove 

>=.100). 

3 

EM.f 

Stepwise 
(Criteria: 
Probability-of- 
F-to-enter  <=  . 
050. 

Probability-of- 
F-to-remove 
>=  .100). 

4 

EM. a 

Stepwise 
(Criteria: 
Probability-of- 
F-to-enter  <=  . 
050. 

Probability-of- 
F-to-remove 
>=  .100). 

a  Dependent  Variable:  Overall_Rating 


Model  Summary8 


Mode 

1 

R 

R  Sou  a  re 

Adjusted  R 
Sauare 

Std.  Error  of 
the  Estimate 

i 

.80 13 

.641 

.639 

418 

a.  Predictors:  (Constant),  EMe 

b.  Predictors:  (Constant).  EM  e.  EM  g 

c.  Predictors:  (Constant),  EM.e,  EM.g.  EM.f 

d.  Predictors:  (Constant).  EM  e,  EM.g.  EM.f,  EM. a 

e.  Dependent  Variable:  Overall_Rating 
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Model  Summary 


Mode 

1 

R 

R  Sou  are 

Adjusted  R 
Sauare 

Stri.  Error  of 
the  Est  mate 

2 

.843° 

.711 

708 

.376 

3 

.882° 

.743 

.738 

.356 

4 

.872° 

.760 

754 

.345 

a.  Predictors:  (Constant).  EM.e 

b.  Predictors:  (Constant).  EM.e.  EM.g 

c.  Predictors:  (Constant).  EM.e.  EM.g.  EM.f 

d.  Predictors:  (Constant).  EM.e.  EM.g.  EM.f.  EM. a 
e  Dependent  Variable:  Overall_Rating 


ANOVA8 


Model 

Sum  of 
Sauares 

df 

Mean  Sauare 

F 

Sip. 

1 

Regression 

50.701 

1 

50.791 

200.919 

.000 

Residual 

28.458 

163 

.175 

Total 

79.248 

164 

2 

Regression 

56  382 

2 

28.191 

199.721 

000° 

Residual 

22.867 

162 

.141 

Total 

79  248 

164 

3 

Regression 

58  861 

3 

19.620 

154.945 

oooc 

Residual 

20.387 

161 

.127 

Total 

79  248 

164 

4 

Regression 

60.244 

4 

15.061 

126.797 

.000° 

Residual 

19.005 

160 

.119 

Total 

79.248 

164 

a.  Predictors:  (Constant).  EM  e 

b.  Predictors:  (Constant).  EM  e,  EM.g 

c.  Predictors:  (Constant).  EM.e.  EM.g.  EM.f 

d.  Predictors:  (Constant).  EM  e.  EM  g.  EM.f.  EM. a 

e.  Dependent  Variable:  Overall_Rating 


Coefficients8 


Unstandardized  Coefficients 

Standardized 

Coefficients 

95%  Confidence  Interval  for  B 

B 

Std.  Error 

Beta 

t 

s«. 

Lower  Bound 

Upper  Bound 

1 

(Constant) 

1  068 

.146 

7.313 

.000 

.780 

1.357 

EM.e 

.689 

.040 

.801 

17.056 

.000 

.609 

.768 

2 

(Constant) 

.734 

.142 

5.182 

.000 

.454 

1.014 

a  Dependent  Variable:  Overall_Rating 
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Coefficients 


Model 

Un standardized  Coefficients 

Standardized 

Coefficients 

t 

Sig. 

95%  Confidence  Interval  for  B 

B 

Std.  Error 

Beta 

Lower  Bound 

Upper  Bound 

2 

EM.e 

.411 

.057 

.478 

7.197 

.000 

.298 

.524 

EM.g 

.353 

.056 

.418 

6.294 

.000 

.242 

.464 

3 

(Constant) 

.324 

.163 

1.988 

.049 

.002 

.646 

EM.e 

.368 

.064 

.463 

7.343 

.000 

.291 

.505 

EM.g 

.300 

.054 

.355 

5.501 

.000 

.102 

.407 

EM.f 

.169 

.045 

.192 

4.425 

.000 

.110 

.288 

4 

(Constant) 

.296 

.158 

1.869 

.064 

-.017 

.608 

EM.e 

.307 

.059 

.356 

5.207 

.000 

.190 

.423 

EM.g 

.166 

.061 

.232 

3.212 

.002 

.075 

.316 

EM.f 

.188 

.044 

.182 

4.315 

.000 

.102 

.275 

EM  a 

.204 

.060 

.257 

3.412 

.001 

.086 

.322 

a.  Dependent  Variable:  Overall_Rating 


e 

Excluded  Variables 


Collinearity 

Statistics 

Model 

Beta  In 

t 

Sig. 

Partial 

Correlation 

Tolerance 

1 

EM  a 

,433d 

6.164 

.000 

.436 

.365 

EM.b 

329d 

4.467 

.000 

.331 

.364 

EM.d 

156d 

2.808 

.006 

.215 

685 

EM.f 

245d 

5.325 

.000 

.386 

889 

EM.g 

,418d 

6.264 

.000 

.443 

.404 

2 

EM  a 

.280° 

3.539 

.001 

.269 

.266 

EM.b 

.205° 

2.824 

.005 

.217 

.323 

EM.d 

.098° 

1.897 

.060 

.148 

.660 

EM.f 

.192° 

4.425 

.000 

.329 

845 

3 

EM. a 

.257° 

3.412 

.001 

.260 

.265 

EM.b 

.190° 

2.748 

.007 

.212 

.322 

EM.d 

.083° 

1.685 

.094 

.132 

.657 

4 

EM.b 

.135° 

1.923 

.056 

.151 

.297 

EM.d 

.056° 

1.146 

.252 

.061 

636 

a.  Predictors  in  the  Model:  (Constant).  EM.e 

b.  Predictors  in  the  Model:  (Constant).  EM.e.  EM.g 

c.  Predictors  in  the  Model:  (Constant),  EM.e,  EM.g.  EM.f 

d.  Predictors  in  the  Model:  (Constant).  EM.e,  EM.g,  EM.f.  EM. a 

e.  Dependent  Variable:  Overall_Rating 
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Residuals  Statistics 


Minimum 

Maximum 

Mean 

Std.  Deviation 

N 

Predicted  Value 

2.16 

4.77 

3.50 

.606 

165 

Residual 

-1.084 

1.111 

.000 

.340 

165 

Std.  Predicted  Value 

-2.203 

2.007 

.000 

1.000 

165 

Std  Residual 

-3.146 

3.225 

.000 

088 

165 

a.  Dependent  Variable:  Ove*-all_Rating 


Charts 


Histogram 


Dependent  Variable:  Overall  Rating 


Regression  Standardized  Residual 


Mean  =6  75E-16 
Sto.  Dev  =0.983 
N  =185 
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Regression  Standardized  Residual  Expected  Cum  Prob 


Normal  P-P  Plot  of  Regression  Standardized  Residual 


Dependent  Variable:  OverallRating 


Scatterplot 


Dependent  Variable:  Overall  Rating 


Regression  Standardized  Predicted  Value 
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Appendix  R:  Recommended  Revised  DD  Form  2626 


PERFORMANCE  EVALUATION 
(CONSTRUCTION) 


11.  CONTRACT  NUMBER 


12.  PROJECT  NUMBER 


IMPORTANT  -  Be  sure  to  complete  Part  III  -  Performance  Evaluation  of  Contractor  on  reverse 


PART  I  -  GENERAL  CONTRACT  DATA 


3.  TYPE  OF  EVALUATION  (X  one) 

IlNTERIM  (List percentaae  %)  1  1 

FINAL 

1  1 AM ENDED 

4.  TERMINATED  FOR  DEFAULT 


6.b.  TYPE  OF  CONTRACT  (X  one) 

FIRM  FIXED  PRICE  |  |COST  REIMBURSEMENT 

| OTHER  (Specify) 


10.  SIGNIFICANT 
DATES 


a.  DATE  OF  AWARD 

b.  ORIGINAL  CONTRACT 

COMPLETION  DATE 

c.  REVISED  CONTRACT 

COMPLETION  DATE 

d.  DATE  WORK  ACCEPTED 

e.  CONSTRUCTION  START  DATE 

f.  BENEFICIAL  OCCUPANCY  DATE 

PART  II  - 

EVALUATOR  INFORMATION 

11.  EVALUATED  BY 

a.  ORGANIZATION  (Name  and  Address  -  Include  ZIP  Code) 

b.  TELEPHONE  NUMBER  (Include  Area 
Code) 

c.  NAME  AND  TITLE 

d.  SIGNATURE 

e.  DATE 

|1 2.  EVALUATION  REVIEWED  BY  | 

a.  ORGANIZATION  (Name  and  Address  -  Include  ZIP  Code) 

b.  TELEPHONE  NUMBER  (Include  Area 
Code) 

c.  NAME  AND  TITLE 

d.  SIGNATURE 

e.  DATE 

13.  AGENCY  USE  (Distribution,  etc.) 
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PART  III  -  PERFORMANCE  EVALUATION  OF  CONTRACTOR 


N/A  =  NOT  APPLICABLE  5  =  OUTSTANDING  4  =  ABOVE  AVERAGE  3  -  SATISFACTORY  2  -  MARGINAL  U  -  UNSATISFACTORY 


PERFORMANCE  ELEMENTS  &  ITEMS 


14.  QUALITY  OF  CONTROL 


QUALITY  OF  WORKMANSHIP 


ADEQUACY  OF  THE  CQC  PLAN 


IMPLENENTATION  OF  THE  CQC  PLAN 


QUALITY  OF  THE  CQC  DOCUMENTATION 


STORAGE  OF  MATERIALS 


f.  ADEQUACY  OF  MATERIALS 


g.  ADEQUACY  OF  SUBMITTALS 


h.  ADEQUACY  OF  TESTING 


i.  USE  OF  SPECIFIED  MATERIALS 


15.  EFFECTIVENESS  OF  MANAGEMENT 


COOPERATION  AND  RESPONSIVENESS 


MANAGEMENT  OF  RESOURCES/PERSONNEL 


ADEQUACY  OF  SITE  CLEAN-UP 


EFFECTIVENESS  OF  JOB-SITE  SUPERVISION 


COMPLIANCE  WITH  LAWS  AND  REGULATIONS 


f.  PROFESSIONAL  CONDUCT 


17.  TIMELY  PERFORMANCE 


ADEQUACY  OF  INITIAL  PROGRESS  SCHEDULE 


ADHERENCE  TO  APPROVED  SCHEDULE 


RESOLUTION  OF  DELAYS 


SUBMISSION  OF  REQUIRED  DOCUMENTS 


e.  IDENTIFICAITON/CORRECTION  OF 

DEFICIENT  WORK  IN  A  TIMELY  MANNER 


f.  COMPLETION  OF  PUNCHLIST  ITEMS 


g.  SUBMISSION  OF  UPDATED  AND  REVISED 
PROGRESS  SCHEDULES 


19.  COMPLIANCE  WITH  SAFETY  STANDARDS 


ADEQUACY  OF  SAFETY  PLAN 


IMPLEMENTATION  OF  SAFETY  PLAN 


CORRETCION  OF  NOTED  DEFICIENCIES 


\mmisammmiiSimiaaBaat 


OUTSTANDING 

ABOVE 

AVERAGE 

SATISFACTORY 

MARGINAL 

UNSATISFACTORY 
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Appendix  S:  IRB  Exemption  Email  from  Lt  Col  Barelka 


Rebecca, 

File  this  email  and  put  in  an  appendix  when  you  get  the  final  draft 
together  later  this  year. 

c  j  w 


- Original  Message - 

From:  Barelka  Alexander  J  LtCol  AFIT/ENV 

Sent:  Thursday,  November  06,  2008  8:38  AM 
To:  West  Christopher  J  LtCol  AFIT/ENV 
Subject:  RE:  IRB  exemption 

If  it's  existing  data  you  just  satisfied  the  IRB  requirement  and  not 
further  action  is  required  on  your  part. 

- Original  Message - 

From:  West  Christopher  J  LtCol  AFIT/ENV 
Sent:  Wednesday,  November  05,  2008  3:04  PM 
To:  Barelka  Alexander  J  LtCol  AFIT/ENV 
Subject:  IRB  exemption 

Alex, 


It's  been  awhile  so  I  don't  know  the  quick  answer  -  have  student  with 
existing  regulatory  required  performance  data  on  construction 
contractor  performance  for  local  Army  Corps  of  Engineers  detachment  - 
all  performance  data  is/will  be  kept  anonymous  and  will  not  be 
presented  in  the  thesis  or  made  available  to  anyone  else.  This 
requires  as  a  formality  an  IRB  exemption  letter,  correct?  If  so  can 
you  point  me  toward  a  copy  and  we'll  do  the  paperwork. 


Chris 
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